Fragile Evidence for an Ideological Bias in the Production of Research Finding: Comment on Borjas and Breznau
Borjas and Breznau (Borjas and Breznau 2026; hereafter referred to as B&B) recently reported that researchers' ideology influences their empirical findings. Although we were able to reproduce their numerical results, a reanalysis shows that the reported association is not robust. Specifically, the association hinges on a coding error. Due to idiosyncratic variable coding, 80 data points from four research teams that contradict the ideology hypothesis were effectively excluded from the analysis. Correcting this error renders the ideology effect statistically insignificant. Additionally, B&B departed from the outcome construction and weighting scheme used in a previous paper by one of the authors based on the same data. Using the same specifications as in the previous paper also renders the ideology effect indistinguishable from zero. Therefore, we conclude that B&B do not provide robust evidence of an ideological bias. Rather, the reported association appears to be a statistical artifact resulting from three questionable modeling decisions.
This paper received a response:
Brojas, G.J. and N. Breznau. 2026. A Lack of Robustness in Robustness Checking from Auspurg and Brüderl. I4R Discussion Paper Series No. 304. Institute for Replication.