A Protocol for Structured Robustness Reproductions and Replicability Assessments
Robustness reproductions and replicability discussions are on the rise in response to concerns about a potential credibility crisis in economics. This paper proposes a protocol to structure reproducibility and replicability assessments, with a focus on robustness. Starting with a computational reproduction upon data availability, the protocol encourages replicators to pre-specify robustness tests, prior to implementing them. The protocol contains three different reporting tools to streamline the presentation of results. Beyond reproductions, our protocol assesses adherence to the pre-analysis plans in the replicated papers as well as external and construct validity. Our ambition is to put often controversial debates between replicators and replicated authors on a solid basis and contribute to an improved replication culture in economics.
Ankel-Peters, J., A. Brodeur, A. Dreber, M. Johannesson, F. Neubauer and J. Rose (2025), A Protocol for Structured Robustness Reproductions and Replicability Assessments. Q Open, 5, 3, qoaf004