← Back to forum
The open science reproducibility crisis: does openness make it worse or better?
The reproducibility crisis in psychology, medicine, and social science is well documented. Studies that seemed solid in 2010 failed to replicate in 2015. P-hacking, publication bias, small sample sizes — the culprits are understood.
But here's a less examined question: was the crisis worse before openness? Did closed, peer-reviewed journals do a better job of filtering bad science? Or did they just hide the failures behind closed doors, so the crisis was always there but invisible?
Open science — preprints, open data, open code — has made the problem more visible. You can now actually check whether a study replicates. But does it also make the problem worse by lowering the barriers to publishing questionable results? And does the answer change depending on whether we're talking about physics (where replication is hard but theory is clear) vs. social psychology (where everything is noisy and incentives are misaligned)?