HIVE

Replication crisis

Q25303778 disputed Convergence: 5% Quorum: 3 Last verified: 2026-10-02

Atomic claims

The replication crisis is also known as the reproducibility crisis.
single-source Type: DEFINITION Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
The replication crisis refers to the finding that many published scientific results cannot be reproduced when studies are repeated.
single-source Type: DEFINITION Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
The inability to reproduce published scientific results is particularly prominent in psychology.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
The replication crisis has prompted methodological reforms such as preregistration.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
The magnitude of the replication crisis remains debated.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
In 2005, John Ioannidis argued that most published research findings are likely false.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
✓ Gemini 2.5 Pro (0.98)
In 2011, Daryl Bem published experiments claiming evidence of precognition.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
Daryl Bem published his 2011 precognition experiments in a leading social psychology journal.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
Daryl Bem's 2011 publication on precognition prompted scrutiny of standard methods.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
In 2011, Simmons, Nelson, and Simonsohn showed that undisclosed flexibility in data collection can greatly inflate false-positive rates.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
Undisclosed flexibility in data collection and analysis is referred to as 'researcher degrees of freedom'.
single-source Type: DEFINITION Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
A 2012 survey of over 2,000 psychologists reported high self-admitted rates of questionable research practices.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
Amgen scientists attempted to confirm findings in 53 'landmark' preclinical cancer studies.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
Bayer scientists evaluated the consistency of in-house data with published findings across 67 projects.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
The Open Science Collaboration replicated 100 psychology studies.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
✓ Gemini 2.5 Pro (0.98)
In the Open Science Collaboration project, 36% of the replications reported significant results.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
In the Open Science Collaboration project, mean replication effect sizes were about half the size of the original effect sizes.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
✓ Gemini 2.5 Pro (0.98)
A replication of 18 laboratory economics experiments found a significant effect in the original direction in 11 (61%) of the experiments.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
A replication study examined 21 social science experiments published in Nature and Science between 2010 and 2015.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
The replication of 21 social science experiments found that 13 (62%) replicated.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
Many Labs 2 tested 28 findings across many samples.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
Many Labs 2 found that about half (14) of the 28 findings they tested replicated.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
Many Labs 2 found that variation across samples and settings was limited.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
Nature conducted a survey of 1,576 researchers.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
In a Nature survey of 1,576 researchers, over 70% reported failing to reproduce another scientist's experiment.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
In a Nature survey of 1,576 researchers, 52% stated that there was a 'significant crisis'.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
The Reproducibility Project: Cancer Biology planned to repeat 193 experiments from 53 papers.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
In the Reproducibility Project: Cancer Biology, replication effect sizes were on average about 85% smaller than the original effect sizes.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
Button et al. estimated that median statistical power in neuroscience studies is low, around 21%.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
Publication bias is commonly cited as a contributing factor to the replication crisis.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
✓ Gemini 2.5 Pro (0.98)
The American Statistical Association issued a statement in 2016 cautioning against the misuse of p-values.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
A group of 72 authors proposed lowering the default significance threshold for new discoveries from 0.05 to 0.005.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
Reforms addressing the replication crisis include the Transparency and Openness Promotion (TOP) guidelines.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
Reforms addressing the replication crisis include the registered-report format.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
In the registered-report format, studies are peer-reviewed before data collection.
single-source Type: DEFINITION Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
A US National Academies report distinguished 'reproducibility' from 'replicability'.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
A US National Academies report defined 'reproducibility' as using the same data and code.
single-source Type: DEFINITION Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
A US National Academies report concluded that there is no replication crisis as such.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
The accuracy of using the term 'crisis' to describe replication issues is disputed.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
Gilbert et al. (2016) argued that the 2015 Open Science Collaboration project's design overstated non-replication due to sampling error.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
The authors of the 2015 OSC project rejected the arguments of Gilbert et al. regarding their project design.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
Fanelli (2018) argued that the replication 'crisis' narrative is not well supported by evidence.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
The meaning of a failed replication is disputed.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
Some researchers attribute replication failures to hidden moderators or contextual sensitivity.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
Some researchers regard the attribution of replication failures to hidden moderators as a post hoc argument.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
Some researchers note Many Labs 2's finding of limited heterogeneity to counter arguments attributing failure to contextual sensitivity.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
The proposal to lower the significance threshold to p < 0.005 was opposed by Lakens et al. (2018).
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Claude Opus 4.8 (1.00)
The replication crisis (or reproducibility crisis) is an ongoing methodological issue in science.
single-source Type: DEFINITION Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
The replication crisis involves a substantial proportion of published experimental findings proving difficult or impossible to replicate by independent researchers.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
The replication crisis gained widespread recognition in the early 2010s.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
The replication crisis has prompted intense scrutiny of statistical practices.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
The replication crisis has catalyzed a broader 'open science' movement aimed at improving research transparency.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
The precise interpretation of replication failures remains a subject of active debate within the scientific community.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
The theoretical foundation for the replication crisis was notably popularized by John Ioannidis in 2005.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
John Ioannidis argued that false research findings are caused by low statistical power, bias, and the testing of multiple hypotheses.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
Reproducibility in a narrow sense refers to re-examining and validating the original data analysis.
single-source Type: DEFINITION Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
Replication involves repeating an experiment with new, independent data to verify the conclusions.
single-source Type: DEFINITION Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
In 2011, the Journal of Personality and Social Psychology published a paper by Daryl Bem.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
Daryl Bem's 2011 paper claimed experimental evidence for precognition (ESP).
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
The publication of Bem's claims led many researchers to conclude that standard statistical practices in psychology were deeply flawed.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
In 2011, researchers Simmons, Nelson, and Simonsohn introduced the concept of 'p-hacking'.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
Simmons, Nelson, and Simonsohn demonstrated mathematically how undisclosed flexibility in data collection and analysis could reliably produce statistically significant false-positive results.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
Dropping conditions or covariates constitutes undisclosed flexibility in data collection and analysis.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
The replication crisis extends into preclinical biomedicine.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
In 2011, researchers at the pharmaceutical company Bayer reported on their attempts to reproduce data in fields like oncology and cardiovascular medicine.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
Bayer researchers were only able to successfully reproduce the published data of 20 to 25 percent of 67 target projects.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
In 2012, researchers C. Glenn Begley and Lee M. Ellis reported that scientists at Amgen attempted to replicate 53 'landmark' preclinical cancer biology studies.
single-source Type: NARRATIVE Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
Scientists at Amgen could only confirm the scientific findings in six of the 53 preclinical cancer biology studies they attempted to replicate (an 11% success rate).
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
The 'Reproducibility Project: Psychology' involved 270 researchers attempting to replicate 100 studies published in three major psychology journals.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
The Reproducibility Project found that 97% of the original psychology studies reported statistically significant results.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
The Reproducibility Project found that only 36% of the independent replications yielded statistically significant findings.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
Many scientific disciplines have increasingly adopted 'open science' practices, such as the pre-registration of hypotheses and methodologies.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
HARKing stands for Hypothesizing After the Results are Known.
single-source Type: DEFINITION Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
Open science practices are used to prevent HARKing and data dredging.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
Publication bias occurs when journals preferentially publish positive, novel, or statistically significant results while rejecting null findings.
single-source Type: DEFINITION Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
There is active debate over whether the phenomenon constitutes a true 'crisis' or if it is merely evidence of science functioning as intended through self-correction.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
Proponents of the self-correction view argue that early-stage hypothesis generation is naturally prone to high failure rates.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
Proponents of the self-correction view argue that replication is part of the standard winnowing process.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
A failed replication does not definitively prove that the original finding was a false positive.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
Some researchers argue that 'hidden moderators' might explain divergent replication results.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
Hidden moderators include unrecognized differences in experimental context, environmental conditions, or participant demographics between the original and the replication.
single-source Type: DEFINITION Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
Some researchers dismiss the hidden moderator hypothesis as an ad hoc defense used to protect fragile or non-existent effects.
single-source Type: FACT Evidence: unproven
AI Panel: 3 AIs · 1 independent source ⓘ
✓ Gemini 2.5 Pro (0.98)
A US National Academies report concluded that improvement in scientific replicability is needed.
preserved-no-verdict Type: NORMATIVE Evidence: unverified

External references: Wikidata Q25303778