PLOS

Over the past several years scientific publishers—including Nature, Science, PNAS, and of course, PLOS—have adopted policies requiring authors to provide the data underlying their manuscript as part of their journal submission, and to make that data publicly available at publication. These Open Data policies are designed to increase public trust in peer reviewed research, and to support reproducibility by improving data accessibility and clarity.

For reviewers asked to assess data as part of their manuscript evaluation, Open Data policies can add a new layer of complexity. What does an effective data review look like, and what should peer reviewers watch out for?

"Numbers are friends for me, more or less." — Wim Klein

Whether you’re reviewing a manuscript, a dataset, or both, your role as a reviewer is the same: draw upon your specific subject-area expertise to make a thoughtful assessment of the work; identify concerns that might make it unsuitable for publication, as well as areas for improvement or clarification. Importantly, reviewers are not expected to do authors’ job for them – only to highlight issues in need of attention. Read more about how to write a peer review.

Here are the five key issues to watch out for when assessing a dataset. As you review the manuscript and associated data, be sure to confirm that:

✓	The data is accessible The data underlying study you are assessing really is available at the url listed.
✓	You can tell what you’re looking at The authors have provided metadata clearly describing the format and content of the data files.
✓	The data you see matches the data referenced in the manuscript The data relates to the study you’re currently reviewing, and each result mentioned in the manuscript is present.
✓	The data presentation makes sense The format, file types, and arrangement of data are clear and appropriate for the type of study and the research that was conducted.
✓	The data itself makes sense The values reported are physically possible and plausible. Results fall within the appropriate range for the phenomenon described. Data points are internally consistent.

That’s it! If, as you perform these checks, you identify a potential concern, be sure make note in the section of your review discussing specific areas for improvement, itemized under Major Issues. Some journals may also have a subsection for data feedback within the reviewer form.

➞ Download the full Reviewer’s Quick Guide to Assessing Open Datasets, developed by PLOS in collaboration with the Cambridge Data Champions.

Download

It’s (Almost) Open Data Day... Join the celebration! Saturday March 2 attend an Open Data event near you, or start one of your own.

Weigh in…
Do you consult publicly posted data when it is available for an article you are reading?

•	Yes - I routinely consult the data for articles I’m reading
•	Sometimes - I occasionally consult the data for articles I’m reading
•	Never - I’ve never consulted the data for an article I’m reading
•	I haven’t seen this yet

Public Library of Science
1160 Battery St. Suite 225
San Francisco, CA 94111
US