‘Rigor Mortis’ Reveals Rampant Sloppiness In Science
via The Global Warming Policy Forum (GWPF)
http://www.thegwpf.com
Biomedical science faces an existential threat — not the threat of budget cuts or anti-science political views, but one from within the discipline itself. So argues Richard Harris in his new book, “Rigor Mortis.”
The book is an alarming and highly readable summation of what has been called the ‘reproducibility crisis’ in science — the growing realization that large swathes of the biomedical literature may be wrong. By one estimate, according to reporting by Harris, up to 85 percent of published studies may be incorrect. By another, unreliable research costs U.S. taxpayers $28 billion per year — an amount equivalent to nearly the entire annual budget of the U.S. National Institutes of Health (NIH).
Harris has been tracking the reproducibility crisis for years as a reporter for National Public Radio. But his book highlights the urgency of the problem, which Harris says threatens to erode public trust in science.
Most worryingly, Harris reports, an untold number of people have participated in drug trials on the basis of demonstrably false findings, putting their own lives at risk.
He highlights an analysis of nine drugs that showed promise for treating amyotrophic lateral sclerosis (ALS) in animal studies. Eight of the drugs failed to slow the deadly neurodegenerative disease in people.
When scientists at the ALS Therapy Development Institute in Cambridge, Massachusetts, explored why the trials were a bust, the problem turned out to be worse than the known issue that animal models don’t faithfully reproduce human ALS. The scientists found a host of flaws in the animal studies themselves. Some studies included too few animals to detect a statistically valid effect; others failed to account for gender differences. Male mice in the ALS model die earlier than females, and in some of the studies, researchers included more females in the treatment group than in the control group — which could make the drug look like it’s working even if it isn’t.
When the Cambridge scientists redid the studies with more rigorous methods, not one of the candidate drugs showed a benefit. The clinical trials had been destined to fail, at a cost of hundreds of millions of dollars.
Publish or perish:
Most scientists don’t intentionally conduct shoddy experiments, Harris notes. They want to produce findings that benefit society. But science has evolved in ways that make it difficult for them to follow that dream. Funding is scarce and the pressure to publish in high-impact journals is intense, he writes. As a result, scientists prioritize studies that are likely to make a splash over less sexy but perhaps more rigorous work.
Through interviews with researchers and journal editors, Harris learns that the pressure to publish and secure precious grant money drives scientists to engage in behaviors that undermine the reliability of their work. These include ignoring data that don’t fit with their theories, massaging statistics and overhyping results. Some scientists fabricate data outright.
Once a flawed study is published, there is no incentive for other scientists to attempt to replicate its results or point out its flaws, because they must spend their time and money developing and publishing their own research if they wish to advance their careers. And there is no way for scientists to admit their mistakes without damaging their reputations.
Watchdogs and U.S. lawmakers have become aware of the problem, spurring funding agencies to address the issue. The NIH, for instance, has spelled out expectations for how grantees should ensure rigor in their research. The rules include authenticating materials such as antibodies and cell lines, and accounting for gender. Dutch authorities announced last year that they would invest 3 million euros to replicate studies.
via The Global Warming Policy Forum (GWPF) http://www.thegwpf.com
April 4, 2017 at 11:35PM
