van den Akker, Olmo, Bakker, Marjan, van Assen, Marcel, Pennington, Charlotte Rebecca, Verweij, Leone, Elsherif, Mahmoud, Claesen, Aline, Gaillard, Stefan, Yeung, Siu Kit, Frankenberger, Jan-Luca, Krautter, Kai, Cockcroft, Jamie, Kreuer, Katharina, Evans, Thomas Rhys, Heppel, Frédérique, Schoch, Sarah, Korbmacher, Max, Yamada, Yuki, Albayrak-Aydemir, Nihan, Alzahawi, Shilaan, Sarafoglou, Alexandra, Sitnikov, Maksim, Děchtěrenko, Filip, Wingen, Sophia, Grinschgl, Sandra, Hartmann, Helena, Stewart, Suzanne L. K., de Oliveira, Cátia, Ashcroft-Jones, Sarah, Baker, Bradley and Wicherts, Jelte (2024). The potential of preregistration in psychology: Assessing preregistration producibility and preregistration-study consistency. Psychological Methods , (In Press)
Abstract
Study preregistration has become increasingly popular in psychology, but its potential to restrict researcher degrees of freedom has not yet been empirically verified. We used an extensive protocol to assess the producibility (i.e., the degree to which a study can be properly conducted based on the available information) of preregistrations and the consistency between preregistrations and their corresponding papers for 300 psychology studies. We found that preregistrations often lack methodological details and that undisclosed deviations from preregistered plans are frequent. These results highlight that biases due to researcher degrees of freedom remain possible in many preregistered studies. More comprehensive registration templates typically yielded more producible preregistrations. We did not find that the producibility and consistency of preregistrations differed over time or between original and replication studies. Furthermore, we found that operationalizations of variables were generally preregistered more producible and consistently than other study parts. Inconsistencies between preregistrations and published studies were mainly encountered for data collection procedures, statistical models, and exclusion criteria. Our results indicate that, to unlock the full potential of preregistration, researchers in psychology should aim to write more producible preregistrations, adhere to these preregistrations more faithfully, and more transparently report any deviations from their preregistrations. This could be facilitated by training and education to improve preregistration skills, as well as the development of more comprehensive templates.
Additional Information: | Copyright © American Psychological Association, 2024. This paper is not the copy of record and may not exactly replicate the authoritative document published in the APA journal. The final article is available, upon publication, at: [ARTICLE DOI here once available] |
---|---|
Uncontrolled Keywords: | preregistration,open science,meta-research,preregistration deviationreregistration template,preregistration template,preregistration producibility,preregistration-study consistency,SDG 4 - Quality Education,SDG 8 - Decent Work and Economic Growth |
Publication ISSN: | 1082-989X |
Last Modified: | 27 Jun 2024 11:53 |
Date Deposited: | 17 Jun 2024 17:00 | PURE Output Type: | Article |
Published Date: | 2024-05-22 |
Accepted Date: | 2024-05-22 |
Authors: |
van den Akker, Olmo
Bakker, Marjan van Assen, Marcel Pennington, Charlotte Rebecca ( ![]() Verweij, Leone Elsherif, Mahmoud Claesen, Aline Gaillard, Stefan Yeung, Siu Kit Frankenberger, Jan-Luca Krautter, Kai Cockcroft, Jamie Kreuer, Katharina Evans, Thomas Rhys Heppel, Frédérique Schoch, Sarah Korbmacher, Max Yamada, Yuki Albayrak-Aydemir, Nihan Alzahawi, Shilaan Sarafoglou, Alexandra Sitnikov, Maksim Děchtěrenko, Filip Wingen, Sophia Grinschgl, Sandra Hartmann, Helena Stewart, Suzanne L. K. de Oliveira, Cátia Ashcroft-Jones, Sarah Baker, Bradley Wicherts, Jelte |