Half of AER Results Fragile

Updated: 2025.08.30 1M ago 1 sources
Researchers audited 17 non‑experimental American Economic Review papers (2013 and 2022/23) with alternative analyses vetted by independent experts. Only about 51% of these robustness tests stayed statistically significant, and average test statistics fell to roughly 70% of the originals. Economists surveyed overestimated robustness but could still pick which papers were most solid. — Prestige economics findings are often fragile, so journals, media, and policymakers should demand robustness maps before relying on single studies.

Sources

The robustness reproducibility of the American Economic Review
Tyler Cowen 2025.08.30 100% relevant
Douglas Campbell et al.’s study of 17 AER papers showing 0%–93% robustness significance rates (mean 51%) and reduced t/z-values (mean 70%).
← Back to All Ideas