Mass Reproducibility in Psychological Science
Concerns about the credibility and reproducibility of published findings have motivated numerous initiatives to improve transparency in psychology. In this article, we present evidence from a large-scale reproducibility initiative conducted in partnership between Psychological Science and the Institute for Replication (I4R). Our project systematically reproduces empirical results published in the journal between 2024 and 2025 using independent research teams. Through a combination of Replication Games and researcher-led initiatives, 67 independent teams examined 44% of the articles (i.e., 64 articles) published during our time frame, documenting data and code availability, checking for coding errors, and deviations from preregistration, and computationally reproducing the main numerical results. We find that the majority of articles provide cleaned data and analysis scripts, though raw data and data-cleaning scripts are shared far less often. Computational reproducibility depends strongly on the starting point: 58.2% of articles are fully reproducible from final analysis data, but only 28.4% are fully reproducible from raw data. Teams identify coding errors or statistical reporting inconsistencies in 63% of articles, although most issues are minor; 40.5% of the articles flagged with coding or statistical reporting issues contain discrepancies with potentially major implications for estimates or interpretation. Preregistration is common (74.6% of articles), yet 84% of preregistered articles deviate from their pre-analysis plan, and nearly half of these deviations go unacknowledged. Our findings provide new insights into the reliability of empirical results in a leading psychology journal and highlight the value of structured mass reproducibility initiatives.