Posts Tagged Reproducibility

Is there a reproducibility “crisis” in biomedical science? No, but there is a reproducibility problem

NOTE: Anyone who has seen several derogatory articles about me on the web and is curious about what the real story is, please read this and this.

Journal of Irreproducible ResultsMost scientists I know get a chuckle out of the Journal of Irreproducible Results (JIR), a humor journal that often parodies scientific papers. Back in the day, we used to chuckle at articles like “Any Eye for an Eye for an Arm and a Leg: Applied Dysfunctional Measurement” and “A Double Blind Efficacy Trial of Placebos, Extra Strength Placebos and Generic Placebos.” Unfortunately, these days, reporting on science is giving the impression that the JIR is a little too close to the truth, at least when it comes to reproduciblity, so much so that the issue even has its own name and Wikipedia entry: Replication (or reproducibility) crisis. It’s a topic I had been meaning to write about again for a while. Fortunately, A recent survey published in Nature under the somewhat clickbaity title “1,500 scientists lift the lid on reproducibility” finally prodded me to look into this question again. Before I get to the survey itself, though, I can’t help but do my usual pontificating to provide a bit of background.

Posted in: Basic Science, Pharmaceuticals, Politics and Regulation, Science and Medicine

Leave a Comment (0) →

Science-Based Medicine 101: Reproducibility

So far I have explained why most research (if not carefully designed) will lead to a false positive result. This inherent bias is responsible for many of the illusionary treatment benefits that we hear about so commonly through the media (whether they’re reporting about CAM or Western medicine), because it is their job to relay information in an entertaining way more so than an accurate manner (i.e. good science makes bad television).Then I explained a three step process for determining the trustworthiness of health news and research. We can remember these steps with a simple mnemonic: C-P-R.

The C stands for credibility– in other words, “consider the source” – is the research published in a top tier medical journal with a scientifically rigorous review process?

The P stands for plausibility– is the proposed finding consistent with known principles of physics, chemistry, and physiology or would accepting the result require us to suspend belief in everything we’ve learned about science to date?

And finally we arrive at R – reproducibility. If the research study were repeated, would similar results be obtained? (more…)

Posted in: Science and Medicine, Science and the Media

Leave a Comment (4) →