Robustness and Replicability in Economics (R2E)
In economics,
most empirical work is based on non-laboratory experimental and
quasi-experimental designs. While in other more standardized disciplines,
replicability mostly is a matter of generalizability across contexts, in
economics it also hinges on the robustness of findings across different
specifications. This project will provide replicability rates in economics by
conducting computational and robustness replications of 30 non-laboratory
studies in economics using the original data set. These studies cover different
empirical methods such as primary data-based Randomized Controlled Trials
(RCTs) and secondary data-based quasi-experimental methods. Complementing these
robustness replications, we will also run surveys with experts in the field to
elicit how they assess replication success or failure, and to obtain
assessments of generalizability across contexts – a frequently discussed
limitation of RCTs in particular.
The main objectives are, first, to define replication success for robustness replications.
We will also develop standardized protocols on how to conduct robustness
replications and standardized forms for reporting the results. Second, we will
compare replicability rates across methods and evaluate differences. The prior
of experts usually is that secondary data-based studies are more prone to
p-hacking, HARKing, and publication bias than RCTs. Third, we will develop an
expert interview-based toolkit to assess robustness replicability and
generalizability across contexts.
Publications
Project start:
01. July 2022
Project end:
30. June 2025
Project management:
Prof. Dr. Jörg Peters
Project staff:
Florian Neubauer,
Julian Rose,
Prof. Dr. Nathan Fiala
Project partners:
Stockholm School of Economics,
University of Connecticut
Principal:
Deutsche Forschungsgemeinschaft