Effect sizes shrink on replication

Updated: 2015.10.07 10Y ago 1 sources
In a coordinated attempt to replicate 100 psychology studies, only about 36% reproduced statistically significant results and the average effect size was roughly half the original. The project used standardized protocols and open materials to reduce garden‑of‑forking‑paths and showed that headline findings are often inflated. — It warns media and policymakers to demand replication and preregistration before building policy or public narratives on single, striking studies.

Sources

PSYCHOLOGY. Estimating the reproducibility of psychological science - PubMed
2015.10.07 100% relevant
Open Science Collaboration (Science, 2015, 349: aac4716) mass‑replication results (significance rate ~36%; effect‑size shrinkage).
← Back to All Ideas