An introduction to peer review

This post is a summary of the slides on How to write a helpful peer review report, presented by Zoë Mullan, Editor-in-Chief, The Lancet Global Health, in April 2019. Original link is here.

Contents

How to write a helpful peer review report – comments to authors

• General considerations
• Checklists and guidelines
• Overall structure
• Section by section

General considerations

DO…

• Be professional and courteous
• Be objective, constructive, and specific
• Read the whole paper before starting (including appendices)
• Follow the journal’s guidelines
• Be consistent with your confidential comments to editors
• Remember that the purpose is to assist the authors in improving the paper

DO NOT…

• Write anything you would not be happy to receive yourself
• Line-by-line edit the paper (copy editors do that)
• Ask the authors to include references to your own work unless absolutely necessary
• Request additional analyses/experiments unless they are essential to support claims made in the current manuscript

Checklists and guidelines

• Check the article type against journal information: “Seminars are disease-oriented clinically focused overviews for the generalist, covering epidemiology, pathophysiology, diagnosis, management, and prevention” – Not appropriate to critique novelty in this instance.
• Refer to reporting checklists – e.g., CONSORT for randomised trials, PRISMA for systematic reviews
• Attend to any journal-specific instructions to reviewers
  • An outline of the conceptual advance over previously published work;
  • A summary of the specific strengths and weaknesses of the paper;
  • Alternative hypotheses that are consistent with the available data

Overall structure

Initial paragraph(s): A succinct overview of the paper as you interpreted it, with thoughts on its contribution to the field as a whole and a top-level summary of strengths and weaknesses

Questions to consider:

• Does the study add to current knowledge?
• Does it challenge existing paradigms?
• Does it matter?

Example:

“The current study is on a topic of relevance and general interest to the readers of the journal. I found the paper to be overall well written and felt confident that the authors performed careful and thorough field and spectral processing. The design of the field campaign combined with several micro-met stations makes the dataset seem quite useful. However, the description of some very important points were inadequate or completely missing. I have very little confidence in one important analysis, and came away with too many questions to be able to recommend this paper without major revision.”

Major comments

Fundamental to understanding the paper.

Start with most concerning issues and work down, or critique in section order.

Example:

“I have several significant concerns about the presentation and general results that should be addressed prior to publication. First, calibration and validation of conductance models. Ideally, I think it would be best to use separate FLUXNET forest sites for calibration and validation of the model. However, it would be acceptable to split the five sites into separate calibration and validation periods. I assume that is what the authors have done, but this is not explicitly stated. The calibration and validation should be done using different time periods that are explicitly stated.”

Minor comments

Confusing sentences, badly drawn figures, incorrect references, technical clarifications.

Example:

• “Line 141: What does the ‘residue’ represent?”
• “Line 352: Please add the range of estimation uncertainty (e.g., 95% confidence interval) to the trend.”
• “Fig. 4, lines 382–388. Something does not seem correct here. The description in the text does not seem to match the figure.”

Section by section

Introduction

• Is the research question clearly stated?
• Is a coherent case made for why the research question is important?
• Are the most relevant pieces of previous work cited?

Methods

• Are methods described clearly enough to be repeated?
• Are they the most appropriate way to answer the research question?
• If humans or animals were involved, is ethics approval reported and is the research ethical?
• Is informed consent necessary and reported?

Results

• Are results reported for all experiments/analyses described in the Methods?
• Are any results reported for which methods have not been described?
• Are they presented clearly and consistently (e.g., do the figures and text match)?

Discussion

• Does the interpretation reasonably reflect the results?
• Are potential limitations presented?
• Do the results conflict with other research and is explanation given?
• Has the research question been answered?
• Have authors summed up what findings mean in the “grand scheme of things”?

Title and abstract

• Does the title reflect the research question and study type?
• Does the abstract contain aim and key methodological details?
• Are findings reported in the abstract consistent with the main text?
• Is anything presented in the abstract that wasn’t in the paper?
• Is the conclusion justified by the data in the paper?
• Are implications for future work stated?

Summary

• Be professional and courteous
• Read the whole paper and refer to guidelines before writing anything
• Structure your report:
  1. Summary of strengths, weaknesses, and contribution
  2. Major comments
  3. Minor comments
• Remember that the goal is to advance knowledge