The Peer Review Myth in Cancer Research
"Peer-reviewed" gets treated like a stamp of truth in cancer treatment and clinical trials discussions, but Dr. Chaplin spent decades in drug development learning that the phrase means far less than most people assume. In this episode, he breaks down what peer review actually checks, the real replication numbers behind published cancer research, and the one filter that separates a durable finding from a story that simply sounded good enough to print.
What Peer Review Actually Checks
Peer review is a plausibility filter, not a truth machine. A handful of busy, unpaid reviewers read a paper and judge whether the methods sound reasonable and the logic holds together. What they are not doing is rerunning the experiment or verifying the raw data. That distinction matters enormously for anyone relying on published cancer treatment research to make decisions.
The Real Replication Numbers
When researchers actually attempt to reproduce published findings, the results are humbling. The Reproducibility Project: Cancer Biology could only replicate roughly 37 to 40 percent of experiments from top journals. The best efforts of the pharmaceutical company Amgen reproduced just 11 percent of landmark preclinical cancer papers. In Jay's own career running confirmation experiments on technology pitched for licensing, the number was closer to one in forty. Even findings that do replicate tend to come back with effect sizes about 85 percent smaller than the original report.
Clinical Trials and the Cell-to-Human Gap
Cell line and animal experiments fail to replicate at roughly the same rate, but the real cliff comes when moving from animal models to actual clinical trials. Only 10 to 20 percent of drugs that enter human trials get approved, and in oncology specifically, some analyses put that number as low as 3 percent. That gap explains why drug development takes so long and why a promising cell-line result rarely translates directly into a usable cancer treatment.
What Independent Verification Actually Means
The episode's central lesson: a failed replication almost never means the original scientists lied. It usually means one lab, running an experiment one time, is a shakier foundation than most people assume. Real independent verification means unaffiliated researchers, with their own reagents and their own skepticism, reaching the same result. A former student replicating their mentor's finding doesn't count. When multiple truly independent labs confirm a result, that's when a claim has earned trust.
The takeaway for anyone navigating cancer treatment decisions: before accepting any claim as settled science, ask whether it's actually been independently repeated, whether it was tested in cells, animals, or people, and how big the effect really was in the replication, not the original headline.
Accurate science saves lives — and it starts with rejecting simple myths in favor of real understanding. Stay curious.
Disclaimer: This content is for educational purposes only and is not medical advice. It does not replace guidance from your healthcare provider. Cancer and treatment decisions are highly individual—always consult your physician or qualified healthcare professional regarding your specific situation.
A full disclaimer is available Terms and Conditions.