Reviewing Registered Reports

Registered Reports are a form of empirical article offered by a number of journals at Wiley in which the methods and proposed analyses are pre-registered, reviewed, and, in most cases, published prior to the research being conducted. If the Stage 1 article passes peer review, the Stage 2 article is then provisionally accepted for publication before data collection commences. This format is designed to minimize publication bias and research bias in hypothesis-driven research, while also allowing the flexibility to conduct exploratory (unregistered) analyses and report serendipitous findings.

The review process for Registered Reports is divided into two stages. At Stage 1 (Study Design), reviewers assess study proposals before data are collected. At Stage 2 (Completed Study), reviewers consider the full study, including results and interpretation.

Guidelines for reviewers

Stage 1: Study design

Stage 1 manuscripts will include only an Introduction, Methods (including proposed analyses), and Pilot Data (where applicable). In considering Stage 1 manuscripts, reviewers will be expected to assess:

  1. The importance of the research question(s), for journals that normally include this as a criterion for acceptance.
  2. The logic, rationale, and plausibility of the proposed hypotheses.
  3. The soundness and feasibility of the methodology and analysis pipeline (including statistical power analysis where appropriate).
  4. Whether the clarity and degree of methodological detail is sufficient to exactly replicate the proposed experimental procedures and analysis pipeline.
  5. Whether the authors have pre-specified sufficient outcome-neutral tests for ensuring that the results obtained are able to test the stated hypotheses, including positive controls and quality checks.

Following Stage 1 peer review, manuscripts will be accepted, offered the opportunity to revise, or rejected outright. The acceptance of a Stage 1 manuscript means that the Stage 2 manuscript will also be eligible for publication pending successful completion of the study according to the pre-registered methods and analytic procedures, as well as a defensible and evidence-based interpretation of the results. The Stage 1 article will be published after it’s accepted, unless under embargo.

Stage 2: Completed study

Following completion of the study, authors will write the Stage 2 article, including Results and Discussion sections. These Stage 2 manuscripts will more closely resemble a regular article format. The manuscript will then be returned to the reviewers, who will be asked to appraise:

  1. Whether the data are able to test the authors’ proposed hypotheses by satisfying the approved outcome-neutral conditions (such as quality checks, positive controls)
  2. Whether the Introduction, rationale and stated hypotheses are the same as the approved Stage 1 submission (required)
  3. Whether the authors adhered precisely to the registered experimental procedures
  4. Whether any unregistered post hoc analyses added by the authors are justified, methodologically sound, and informative
  5. Whether the authors’ conclusions are justified given the data

Reviewers at Stage 2 may suggest that authors report additional post hoc tests on their data; however, authors are not obliged to do so unless such tests are necessary to satisfy one or more of the Stage 2 review criteria. Please note that editorial decisions will be based on adherence to the approved protocols and experimental design in Stage 1 and conclusions supported by data (even if they are negative findings) as opposed to novelty and perceived importance of results.

Once published, the Stage 1 and Stage 2 articles will be clearly linked within the journal.