Replication Crisis as Procedure Failure

Updated: 2026.04.24 1M ago 6 sources
Reframe psychology’s replication crisis not as a need for new grand theories but as a crisis of research procedures, incentives, and institutional norms (publication bias, low power, p‑hacking, weak peer review). Fixes should prioritize mandatory provenance, routine robustness maps, preregistration, data/analysis audit trails, and changes to hiring/promotion incentives rather than speculative theoretical revolutions. — This reframing shifts oversight and funding toward concrete governance reforms (journals, funders, universities) and away from abstract theory battles, altering how policymakers, educators and funders allocate attention and resources.

Sources

The Problem with Psychedelic Research
Kristen French 2026.04.24 80% relevant
The article documents a systematic procedural weakness (blinding failure) that can inflate apparent treatment effects in psychedelic trials; this is an instance of replication/procedure issues undermining the credibility of a whole research subfield — exactly the kind of mechanism the existing idea names. It cites a JAMA Psychiatry review (24 studies) and an expert (Balázs Sizgeti) linking methodological failure to overstated efficacy claims and policy pressure (Trump executive order).
A Very Unscientific History of Scientific Hoaxes
Jake Currie 2026.03.31 68% relevant
Piltdown and the modern cases illustrate how procedural and methodological weaknesses — and not just bad actors — let false claims persist until new methods or scrutiny expose them, linking to broader replication and verification problems.
How ‘Tiny Shortcuts’ Are Poisoning Science
Eric Neumayer 2026.03.24 90% relevant
The article documents and diagnoses the same mechanism captured by this idea: small, routine researcher choices ('tweaking', p‑hacking, specification searching) systematically inflate published effects and produce non‑replicable results; it names the practice, cites replication evidence (effect sizes shrink, replications often fail), and argues these procedural failures drive a credibility crisis.
One Weird Trick to Get Significant Results
Cremieux 2026.03.13 90% relevant
The article documents a procedural mechanism (subgroup mining / interaction fishing) that produces fragile, non‑replicable results and quantifies why (interactions need far more power), directly illustrating how routine research practices produce the replication crisis.
Psychology’s Greatest Hits (Part 3/3)
Josh Zlatkus 2026.03.11 82% relevant
By stressing that domain‑specific, falsifiable claims and tighter measurement (neuroscience, perceptual science, learning laws) survive scrutiny, the authors diagnose replication failures as methodological/procedural rather than purely cultural, aligning with the idea that replication problems reflect procedural weaknesses.
Psychology’s Greatest Misses (Part 1/3)
Josh Zlatkus 2026.01.07 100% relevant
The article explicitly contrasts Kuhnian‑style revolution with a 2008‑style model failure and urges procedural remedies; it cites education (3‑cueing/phonics) and clinical psychology as domains harmed by flawed methods.
← Back to all ideas