Preregistration and Registered Reports
Guidance on study preregistration and Registered Reports.
When you preregister (or create a pre-analysis plan) you specify your research plan and publicly register it in a repository in advance of undertaking your study e.g., the Open Science Framework; see the UKRN primer for a list of repositories by discipline). The preregistration is a time-stamped, read-only plan (see some examples). When this plan is extended and undergoes formal peer review at a journal ahead of the research, it is called a Registered Report. If they meet the required standards, Registered Reports are accepted by the journal before the research is conducted, independent of the results (“in-principle acceptance").
- There is inherent flexibility in the process of conducting, analysing and writing up research. So called “researcher degrees of freedom (e.g., decisions about outliers, conducting sub-group analyses, etc) create a garden of forking paths where a series of small decisions can opportunistically influence the research outcome and produce a neater, more publishable but less accurate narrative. Detailing, justifying and declaring your decisions before you begin your research, choosing which path to take before setting out, limits your researcher degrees of freedom, and brings the focus off outcomes and on to the process of research
- The primary goal of preregistration is to be transparent about your research process, and to avoid poor research practice. Others can compare your preregistered plans to the final study and evaluate the evidential value of the research, particularly in relation to the ability of your analysis plan to falsify predictions and control for type 1 errors (i.e., incorrectly concluding that an effect exists). For more information see Lakens (2019) (PDF)
- An additional benefit of specifying your research a priori (i.e., independent of the results of the research) is that it encourages you to formulate precise research questions and fine tune your design prior to conducting the study
- It gives you a time-stamped record of your study plans, allowing you to establish priority of ideas earlier in the research process.
When can you preregister?
- Before you start data collection, or data synthesis such as a systematic review or meta-analysis
- When you have been asked to collect more data in peer review
- Before you begin analysis of an existing data set.
A guide to writing a preregistration
Preregistration involves submitting your study plan to a public repository (e.g., the Open Science Framework; see the UKRN primer for a list of repositories by discipline).
The goal is to create a transparent plan, ahead of beginning your study/accessing existing data. So, your preregistration will include details about the study, hypotheses (if you have them), your design, sampling plan, search strategy (for reviews) variables, exclusion and inclusion criteria, and the analysis plan.
The Open Science Framework offers a number of help guides that walk you through the process from a technical perspective.
The more challenging aspect is defining your research plan prior to conducting the study.
If you are conducting confirmatory research (i.e., hypothesis testing work) you need to think about controlling your type 1 error rate and reducing the number of decisions made after data collection (e.g., specifying when you’ll stop collecting data, planning what data you’ll exclude etc). Your hypotheses should also map closely to your research question and analysis plan.
The key to a good preregistration is detail. Your preregistration should be specific, and detailed enough to constrain your ability to make post-data/study decisions.
There are a range of preregistration templates tailored to various research types and disciplines, including qualitative research and secondary data analysis. Select the appropriate template for your research.
Note that for purely exploratory research, preregistration is arguably not required because the research is by its nature unconstrained.
Example preregistrations can provide useful guidance and can be found on the OSF website or by searching on OSF registries. You can filter to find specific categories of preregistration, for example, a search that isolates preregistered qualitative studies.
You can also explore some examples of published preregistered work on Zotero.
You can find help from the preregistration community support who have all preregistered before, and can help answer questions.
When you submit your preregistration to the repository, you will be asked to choose between making the preregistration public immediately or entering it into an embargo.
If you are concerned about making your preregistration public ahead of conducting your research, you can embargo it and keep the document private until a specified date. You should make your preregistration public when the research is published, if not before.
If your registration is embargoed, you can create an anonymised view-only link that allows you to share the private contents with non-contributors. This is particularly useful in blinded peer review, and will allow reviewers to compare your preregistration to your completed research.
Preregistrations are “a plan, not a prison” and you can transparently update your preregistration* if there is an unexpected and necessary change to your study design (e.g., the analysis changed because the data did not meet the assumptions of the planned analyses). In the update, you should explain why the update was necessary, including a description of the change, the rationale for the change, the stage at which the change occurred (e.g., pre or post data collection), and the anticipated impact on the study.
* The OSF has functionality to support the updating of preregistrations. On other platforms that do not support this functionality (e.g., AsPredicted), you can create a new preregistration and explicitly state that it is an update to your original preregistration, and link to that preregistration.
Once the study is complete, use this checklist of items to include when you write up the results of your preregistered research.
Transparently report any deviations from the preregistration, even if you have updated your preregistration. Remember, transparency is key. Include a link to your preregistration (and if applicable, to your updated preregistration) in your paper.
- Preregistration: A pragmatic tool to reduce bias and calibrate confidence in scientific research (Hardwicke & Wagenmakers, 2021)
- Ensuring the quality and specificity of preregistrations (Bakker et al., 2020)
- Seven selfish reasons for preregistration (Wagenmakers & Dutilh, 2016)
- Eight myths about prereg and why you should do it anyway (Corker, 2018)
- Preregistration is hard, and worthwhile (Nosek et al., 2019)
- Preregistration: A plan, not a prison (DeHaven, 2017)
- The value of preregistration for psychological science: A conceptual analysis (PDF) (Lakens, 2019)
- Is pre-registration for you? (Farran, 2020)
- Research preregistration 101 (Lindsay, Simons, & Lilienfeld, 2016)
- The preregistration revolution (Nosek et al., 2018)
- Meta-analyses in times of open science: Practical recommendations from my first meta-analysis (Geiger, 2021).
- Preregistration frequently asked questions
- Help documents and instructions to preregister on the OSF
- Contact Preregistration Community Support for more bespoke help
- Preregistration forms and templates
- Selecting the right preregistration template (PDF) - aimed at educational researchers by broadly relevant (Fleming, 2021)
- Primer on preregistration and Registered Reports (UK Reproducibility Network, 2020)
- Checklist of items to include when creating an analysis plan for some common statistical models
- Information on updating your preregistration
- Introduction to open scholarship and preregistration (Evans, 2022)
- Four stages of embracing preregistration (Morey, 2018)
- Harnessing the benefits of pre-registration for non-experimental studies: Personal experience and examples from psychological research (Dewitte et al., 2021)
- Presentations on preregistration in Economics (PDF), Political Science (PDF), and observational research from Berkeley Initiative for Transparency in the Social Sciences.
Most articles on preregistration consider it to be a positive measure, but there are alternative perspectives:
- Does preregistration improve the credibility of research findings? (Rubin, 2020)
- Paths in strange spaces: A comment on preregistration (Navarro, 2020)
- Preregistration of modeling exercises may not be useful (MacEachern & Van Zandt, 2019).
The Registered Reports model is an extension of preregistration, and refers to a type of research article. In this format, manuscript writing and review occurs in two stages:
Peer review of the study protocol (including rationale, methods, and analysis plan) occurs before the research is conducted (Chambers, 2019). The decision to publish is made before the study is run and based on the importance of the research question, and the rigour of the methods. Authors receive in-principle acceptance, a commitment from the journal to publish the study irrespective of the results.
With an in-principle acceptance in hand, you can complete your study knowing that it will be published regardless of the results (provided you follow your Stage 1 protocol). Once the study is complete, the Stage 2 manuscript is written and sent for review. At this point, the reviewers check whether you have adhered to your Stage 1 plans, and whether your conclusions reflect the data.
Why publish a Registered Report?
Registered Reports have all the benefits of preregistration plus some additional advantages:
- By making the publication decision results blind, Registered Reports take the focus off outcome-based decisions and provide a powerful antidote to publication bias (i.e., the bias whereby the outcome of the study influences the decision to publish, typically in favour of significant effects) while freeing authors from the pressure of producing ‘positive’ or novel results (Chambers and Tzavella, 2020)
- Peer review occurs before the study is run and when those reviews can really improve the quality of the study
- Publication is guaranteed independent of outcome (as long as you follow your Stage 1 plans)
- Receiving in-principle acceptance at Stage 1 means that you can add the paper to your CV earlier compared to waiting for a standard paper to be published. This is particularly important for early career researchers (ECRs).
See all theoretical benefits of Registered Reports and preregistration compared to non-preregistered work (PDF).
A guide to writing a Registered Report
Start by selecting a journal that publishes Registered Reports from this “participating journals” list (or submit to PCI-RegisteredReports) and check the author guidelines.
Complete this template that guides you through the key elements of a Registered Report. Once completed, you have the basis of your Registered Report.
Look at examples of Stage 1 Registered Reports using quantitative methods, qualitative methods, existing data, or for a systematic review/map. Find others by searching for “Registered Reports” in OSF Registries.
Extend the template into a full Stage 1 Registered Report that includes an introduction, method section and analysis plan.
Submit the Stage 1 manuscript for Stage 1 review at your chosen journal. These reviews focus on the importance of the research question and the rigour of your methods.
If you receive in-principle acceptance from the journal, the Stage 2 manuscript will be published irrespective of your findings (e.g., whether the hypothesis is supported or not).
Once in-principle acceptance has been granted, you can conduct your research, write up your Stage 2 manuscript by appending your results and discussion sections to your Stage 1 manuscript, and submit back to your original journal for Stage 2 review. Any minor deviations from the approved Stage 1 manuscript should be transparently reported in the text. More substantial deviations should be flagged with the editor as soon as they arise, and failure to do so could result in rejection of the Stage 2 manuscript. Stage 2 reviews focus on whether the Stage 1 protocol has been followed, and whether the conclusions are justified given the data.
Explore some examples of published Stage 2 Registered Reports.
Registered Report resources
- Registered Reports frequently asked questions
- Up-to-date list of participating journals, author guidelines, and Registered Reports information portal
- Ten reasons to write Registered Reports (now) (Henderson, 2020)
- Blog and Twitter thread on writing Registered Reports as PhD student or ECR
- Templates for meta-analysis Registered Reports (Feldman).
- The past, present, and future of Registered Reports (Chambers & Tzavella, 2020)
- An excess of positive results: Comparing the standard psychology literature with Registered Reports (PDF) (Scheel, Schijen & Lakens, 2021)
- Discussion on the concept, process and effectiveness of Registered Reports from Metascience 2021
- Registered Reports a method to increase the credibility of published results (PDF) (Nosek & Lakens, 2014)
- Primer on Peer Community In Registered Reports (a publishing model that operates at a supra-journal level; Dienes. 2021)
- Mapping the universe of Registered Reports (Hardwicke & Ioannidis, 2018)
- Opening the door to Registered Reports: Census of journals publishing Registered Reports (2013–2020) (Montoya, Krenzer, & Fossum, 2021)
- Improving pedagogy through Registered Reports (McAleer & Paterson, 2021).
Which types of research can be preregistered?
Preregistration can be used for a variety of different research designs and methods, including:
- Quantitative hypothesis testing research (Bosnjak et al., 2021)
- Qualitative research (Haven & Van Grootel, 2019)
- Exploratory research (Dirnagl, 2020)
- Evidence synthesis/reviews (i.e., scoping reviews, reviews of qualitative studies, meta-analysis, or any other type of review) (Stewart et al., 2012; Topor et al., 2020; Van den Akker et al., 2020 ) [template]
- Secondary/pre-existing data (Van den Akker et al., 2021; Weston et al., 2021; Mertens & Krypotos, 2019)
- Applied research (Evans et al., 2021)
- Experience sampling (Kirtley et al., 2019)
- Mathematical/cognitive modelling (Crüwell and Evans, 2019)
- Clinical trials – using established international registries.
Explore preregistration templates for a range of different types of research.
Do preregistration and Registered Reports work?
Work that is preregistered is not automatically better than work that isn’t. But the fact that the research is transparent is necessarily better in the sense that we can check the quality of the work. There is also a greater probability of errors being detected and corrected. In this way, preregistration works.
In terms of evidence for a more direct effect on research quality, we would look for a lower prevalence of “positive” findings, implying that hypotheses had not been adjusted post hoc to fit the data, and that preregistered studies are a more faithful representation of the research conducted. Initial meta-scientific research suggests that this is the case: The proportion of ‘positive’ results in preregistered psychological research (66%; based on preliminary data), in preregistered clinical trials, and in Registered Reports (PDF) (44%), is lower than that in non-preregistered research (PDF) (96%).
For preregistration to be truly effective, researchers need to create clear, precise preregistrations and then adhere to their plans. Research looking at (non-reviewed) preregistrations in economics and political science, and psychology suggests that this is not always the case. But checking both the precision of and adherence to the preregistered plan is central to the Registered Reports review process.
Watch a video discussing the effectiveness of preregistration “Preregistration in the Social Sciences Empirical Evidence of its Effectiveness” from Metascience 2021.