http://www.bmj.com/cgi/content/full/336/7643/545

It purports to show that acupuncture is an effective adjuvant for in vitro fertilization. An accompanying noncritical editorial is here:

http://www.bmj.com/cgi/content/full/336/7643/549

Although it is a SR and MA, point estimates are given and the individual trials are cited, so it may be possible to subject the claim to quantitative Bayesian analysis. I would love for someone to take a stab at it.

And, if you want to use Dr. Sampson’s scale of credibility, consider that BMJ publishes many quackery promoting articles—more than any journal I know except for those that are dedicated to CAM, thus earning it a place on Steve Barrett’s list of nonrecommended periodicals:

http://www.quackwatch.com/04ConsumerEducation/nonrecperiodicals.html

I posted concerning this earlier today:

http://doctorrw.blogspot.com/2008/03/latest-woo-from-bmj-acupuncture-as.html

The recent stoppage of the trial on better control of blood sugar to what is considered “normal” is a good example. The fundamental hypothesis of the trial was that elevated blood sugar is a sign of dysregulation in glucose physiology, and doing things to keep blood sugar in a narrow “normal” range will improve health. What the trial showed is that it is not that simple. What the trial showed was that every instance of elevated blood sugar is not detrimental. We already knew that (or should have known) because the only time blood sugar is constant is at rest. Under stress, even in fasted individuals blood sugar goes up. Is that “dysregulation”? No, it isn’t. It is the body mobilizing glucose reserves and releasing them to peripheral tissues in anticipation of increased need. If the body doesn’t get glucose to those peripheral tissues, those tissues will experience glucose depletion and ATP depletion and will be unable to perform their called upon functions and the consequences will be adverse.

Is the elevated blood sugar of the metabolic syndrome pathological or adaptive? A good question since the “cause” of the metabolic syndrome remains unknown. Is it even dysregulation? There are two types of dysregulation, good regulation around a bad setpoint, and bad regulation around a good setpoint. I suspect that diabetes type 2 is good regulation around a bad setpoint and that diabetes type 1 is the consequence of bad regulation around a good setpoint. You can have both simultaneously. Bad regulation has both positive and negative deviations. A negative deviation in blood sugar, due to too much insulin, can be promptly fatal. The ablation of the pancreatic islet by the immune system may be a mechanism to prevent this. Protective in moderation, fatal when carried to an extreme.

The Bayesian approach begs independent validation, something that breaks the circularity of reasoning that Infofile alludes to. The quite frequent negative trials of alternative methods help supply that. In fact, the naive EBM approach sort of works if this bias towards positive results is factored in, but asymptotically approaching truth only after very many studies have been performed and analysed. This is a waste of resources partly because the inevitable positive studies sustain belief in some.

I have a proposed rule of thumb for meta-analyses: if the results are negative, you can probably believe them; if the results are positive, withhold judgment until you have looked carefully at the quality of the studies and at prior plausibility. And even when they pass those tests, withhold judgment!

Beyond that, I don’t think multiplying by zero, even in the case of homeopathy, would be very useful at convincing anyone. If you start off by saying that it can’t work, of course your result will be a probability of zero. It should be fair to at least give them the one-in-a-google shot that humanity has been collectively hallucinating about the scientific findings of the last few centuries.