Articles

Evidence Thresholds

Defenders of science-based medicine are often confronted with the question (challenged, really): what would it take to convince you that “my sacred cow treatment” works? The challenge contains a thinly veiled accusation — no amount of evidence would convince you because you are a nasty skeptic.

There is a threshold of evidence that would convince me of just about anything, however. In fact, I have been convinced that many scientific claims are likely to be true — sufficiently convinced to act upon the conclusion that they are true. In medicine this means that I am convinced enough to use them as a basis for medical practice.

There are many functional differences between practitioners of SBM and those who accept claims and practices that we would consider to be pseudoscience or fraud, but I was recently struck by one particular such difference — where we set the threshold of evidence before accepting a claim.

Last week I took part in a debate about the legitimacy of homeopathy (you can read my full account here and here). On the other side was Andre Saine, a Canadian naturopath and Dean of the Canadian Academy of Homeopathy. If I had to summarize the key difference between Saine’s position and my own during the debate it was that he accepted incredibly poor evidence as sufficient to establish the reality of homeopathy. His threshold of evidence was astoundingly low.

At the same time he expressed disbelief that skeptics could maintain their skepticism in light of the evidence he presented. I could come to no other conclusion than that Saine simply has no concept of the usual threshold of evidence for acceptance in medicine and in mainstream science generally.

What Would It Take?

What convincing evidence looks like is something we have thought about and written about extensively. I also write about scientific topics outside of medicine, and this has helped give me a broader perspective on this question as well. Proponents of ESP, for example, also accept a lower standard of evidence.

The question is – at what point should the scientific community generally accept the reality of a phenomenon? This means that the alternate explanation, that the positive data is flawed or misleading in some way, can be confidently ruled out.

Here are the four criteria that need to be simultaneously met in order to be considered compelling scientific evidence:

1- Methodologically rigorous, properly blinded, and sufficiently powered studies that adequately define and control for all relevant variables (confirmed by surviving peer-review and post-publication analysis).

2- Positive results that are statistically significant.

3- A reasonable signal to noise ratio (clinically significant for medical studies, or generally well within our ability to confidently detect).

4- Independently reproducible. No matter who repeats the experiment, the effect is reliably detected.

What we often see with dubious medicine (like homeopathy) is that only criteria 2 is necessary – any study that shows statistical significance is taken as iron-clad.

We also often see a shell game similar to buying a new car. Car salespeople often use the four square method – they divide a sheet of paper into four squares – in one is the price of the car, in the other is the loan rate, in the third is the down payment, and in the fourth is the money they will give you on the trade-in. These will all calculate to your monthly payment.

But here’s the trick – the car dealer will use this method to make sure they make their profit. If they give you a good deal on the trade-in, then they don’t give you a good deal on the price of the car. You can never get a good deal on all four squares at the same time.

Proponents of dubious science work the same way – they offer studies that have one or maybe two of the above criteria, but never all four at the same time. They may offer a poorly designed study with positive results, or a well designed study with positive results, but clinically insignificant and unable to be replicated.

There is a reason why you cannot get all four criteria at the same time – because the phenomenon in question is not real. Only a real effect would show up consistently in highly rigorous studies.

It also needs to be pointed out that meeting these criteria is the baseline for scientific acceptance, without even considering prior plausibility. Within each criterion there is a range of quality – how rigorous a study, replicated how many times, with what effect size? The more implausible a claim, the greater the threshold should be in order to overcome that implausibility.

Homeopaths and proponents of implausible claims don’t like this reasoning. They deride it as “plausibility bias.” Everyone else calls it “science.”

But it is important to point out that even without considering prior probability or plausibility, homeopathy still fails to meet even the minimal scientific criteria for acceptance. It is not even close – even if we give it every benefit of the doubt.

Defending the Threshold

If you are convinced by the reality of something like homeopathy, acupuncture, energy medicine, colonic therapy, or something equally unlikely, the threshold of acceptance seems unfair and broken. It seems like a trick nasty skeptics use to deny the reality of your fabulous medicine.

It is, rather, the standard threshold of acceptance within mainstream science (obviously there is a range within that standard, but it is at least a minimal threshold).

Part of the philosophy of science-based medicine is that such a rigorous standard is justified and necessary, and in fact it needs to be even higher than perhaps is currently practiced in medicine. We have discussed many reasons why this is the case, each one can be the topic of a lengthy article, but I will just summarize them here.

- Medical research is challenging as people are generally a variable and noisy system in which to conduct studies and control variables.

- Placebo effects are multifarious and difficult to account for completely.

- Researcher degrees of freedom make it possible to manufacture positive results even out of a completely non-existent phenomenon. This requires special rigor in study design and execution to avoid, and also independent replication.

- Most published studies are wrong, because most new ideas in medicine do not pan out, and most studies are preliminary and are therefore subject to significant researcher bias in the positive direction.

- There is occasional fraud in scientific research.

- Publication bias distorts the overall scientific literature.

- There is considerable financial bias in medical research, as this is an applied science and often billions of dollars ride on the results of research.

- Humans generally are subject to a host of cognitive biases, heuristics, logical fallacies, errors in memory and perceptions, and other mechanisms of self-deception. We can naively be led to believe almost anything with utmost confidence.

Conclusion

Rigorous science anchors us to reality. Without it our beliefs will drift off into a fantasy world that caters to our emotions and desires but has little connection to reality. We might end up believing that pure water can retain the “essence” of a substance that was once diluted in it, and that this essence can heal people based upon completely unrelated features, such as their personality type.

People, left to their own devices, will believe in magic. Such a tendency is our evolutionary inheritance. But so is the capacity for logic and critical thinking.

Over the past two centuries, as scientific medicine has matured, we have learned to apply greater and greater rigor to the study of medicine and disease. We have learned more about our capacity for self-deception, and the subtle ways to manipulate data and research.

We now know what it takes to prove that something is really real, and doesn’t just seem real. We should vigorously resist those who wish to discard this hard-won wisdom because it threatens their cherished belief.

Posted in: Homeopathy, Science and Medicine

Leave a Comment (48) ↓

48 thoughts on “Evidence Thresholds

  1. Eugenie Mielczarek says:

    Steve,
    Thank you for this excellent post. My first criteria is does the claim violate the laws of physics and chemistry?. Too many – grants especially those from NCCAM pay investigators to set up trials testing homeopathy, distance healing,and low energy electromagnetic fields. Usually a back of the envelope calculation by a second year undergraduate physics or chemistry major can demonstrate the impossibility of the ‘hoped for’ positive magic. The recent furor over the claims for “electrified spoon in the mouth” cure for cancer is a case in point.
    Unfortunately grant officers are not able to recognize violations of science.
    Eugenie Mielczarek

  2. cervantes says:

    Also very important: You can’t judge a conclusion only by the p values and quality of experiments testing a particular phenomenon, because the impact of the evidence on our beliefs also depends on prior probability, or plausibility if you prefer the term. Even a well-designed study that results in a statistically significant observation that is inconsistent with the existing edifice of scientific understanding is likely a fluke. Homeopathy is entirely contrary to our understanding of physics, chemistry and biology. Therefore it would require a much higher, not a lower, standard of evidence to accept that a homeopathic remedy is effective, than it would for a remedy with prior plausibility.

    Proving an effective homeopathic remedy would in fact require us to overturn our fundamental understanding of the universe, and look for entirely new physical principles. That would be quite something indeed.

  3. BobbyG says:

    Cited on my REC blog. :)

  4. David Gorski says:

    Actually with respect to plausibility bias, which I’ve written about here:

    http://www.sciencebasedmedicine.org/index.php/plausibility_bias/

    I do think that certain forms of implausibility can trump almost all other evidence. For instance, as I listed in a slide in the talk I gave at NCAS two and a half weeks ago, there are things that trump any clinical trial evidence. Here’s how I put it in the slide:

    We can’t reject something out of hand just because we don’t understand how it could work.

    We can reject a treatment if its proposed mechanism violates scientific principles that rest on far more solid evidence than equivocal, bias-and-error-prone clinical trials:

    • Energy medicine (reiki, therapeutic touch, etc.)
    • Claims based on non-existent anatomical structures (reflexology, iridology, acupuncture, chiropractic subluxations)
    • Non-existent physiologic functions (craniosacral rhythms, movement of bones in neurocranial restructuring, detoxification with coffee enemas or foot baths, dissolving tumors with orally administered pancreatic enzymes)
    • Claims violating multiple laws of physics (homeopathy)

    Another way I like to put it is that, if a proposed therapy fits under any of the above categories, the evidence for it must be at least as compelling as the evidence that says it is impossible. For example, to “prove” homeopathy might work to me, you’d have to show me evidence that is at least in the same order of magnitude in terms of quality and quantity as all the evidence from physics, biochemistry, and physiology that tell us that it can’t work. That would be quite a lot of evidence, and a few methodologically challenged clinical trials showing barely statistically significant effects above placebo don’t cut it.

    All of this is why I never actually use the word “impossible” with respect to, say, homeopathy, but I do say that it’s as close to impossible as I can imagine getting and that, for homeopathy to work, we’d have to be not just wrong but spectacularly wrong in our understanding of the physics and chemistry that tell us it can’t work. That’s possible (remotely), but incredibly unlikely.

    On the other hand, for mechanisms that aren’t so close to impossible, I tend to give more of the benefit of the doubt. Of course, in CAM, this is only true for areas that CAM has appropriated and rebranded, as it has rebranded pharmacognosy as herbalism and supplements, exercise and diet as being somehow “alternative,” etc. It’s quite plausible that herbs could contain drugs and that exercise and diet could affect the course of a disease. Of course, these things are (or should be) science-based medicine.

  5. Janet says:

    I think I will have the numbered points of this post made into a wallet-size card so I can carry them around with me. It would be nice to pull it out when one of these topics comes into the conversation–as it so often seems to do. It might go a ways to stop the “science doesn’t know everything” retort.

    I can also pull it out it when they catch me pushing all the homeopathy bottles behind the other OTC products. :-)

  6. BKsea says:

    The funny thing about homeopathy is that it easily lends itself to large well-designed trials that would provide the necessary evidence if it worked. Successful trials would be incredibly lucrative for the homeopathy industry. The fact that no one even attempts such trials says a lot about homeopathy.

  7. “2- Positive results that are statistically significant.

    3- A reasonable signal to noise ratio (clinically significant for medical studies, or generally well within our ability to confidently detect).”

    I am not sure I follow this. How does clinical significance (i.e. difference between the group getting the treatment and the group getting a placebo is large enough to be of clinical value to the patient) tie into “reasonable signal to noise ratio”? Isn’t the latter more related to the second point (i.e. an attempt to distinguish results that are likely to be obtained if there was no difference from results that are unlikely to be obtained if there was no difference) than clinical significance?

  8. Emil,

    Statistical significance is used to determine if the results are likely to be due to random chance, and it assumes, in fact, that the results are random. P-value, in fact, is the probability of the results being what they are or greater assuming the null-hypothesis and perfect research methodology without any bias or systematic error.

    Clinical significance or reasonable signal to noise ratio addresses an entirely different issue – the possibility of systematic bias in the methods.

    A tiny effect size (relative to the precision of what is being measured), even if fantastically statistically significant, may be due to a very subtle systematic bias. It would be harder to miss a systematic bias that is profound enough to produce a large signal relative to noise. Noise, in this context, is not statistical noise, but all the variables that are not specifically being accounted for. Randomization is the method used to hopefully average out any variables not specifically counted. But there are lots of ways for these variables to still sneak in (such as differential drop out rates), or differences in placebo effect from incomplete blinding, etc.

    In short – statistical significance is about statistical noise in an unbiased random system.
    Signal to noise ratio (clinical significance in medicine) is about the probability that the results might be due to a biased system – to systematic bias.

  9. Steve C says:

    I’d like to respectfully leave a challenge to one of the sentiments of Dr. Novella’s post which sums up the .01% of the general tone and content of SBM and much of the skeptical community in general that I haven’t been able to agree with so far. It’s the last paragraph:

    “We now know what it takes to prove that something is really real, and doesn’t just seem real. We should vigorously resist those who wish to discard this hard-won wisdom because it threatens their cherished belief.”

    I agree with the first sentence but not the last. Stating that we should vigorously resist actual other human beings, I think, misses the point. What we should be resisting is not the people, but rather the beliefs themselves and more particularly the thought processes that make the holding of those beliefs possible–and even more particularly the socially acceptable, culture-based paradigms that create and sustain the thought processes. This is a particularly difficult and often Sisyphean task because we are dealing with the surgical removal of an entire world view, not just a particularly goofy thought. When Dr. Novella tells someone like Andre Saine that magic water is absurd, what he’s really saying, from Saine’s point of view, is something like, “Your entire way of looking at the world is incorrect. The way everybody you know and love and respect looks at the world is incorrect. The way your patients and clients and colleagues and friends look at the world is incorrect. The way the editors and writers of the journals and books you read, and the historians who chronicle the effects of not just homeopathy but all other non-traditional healing methods, are also incorrect. And deeper — society itself is incorrect, because people exchange cold hard cash, billions of dollars of it, for your crazy alternative offerings, so the way the economy works must be flawed somehow, because it supports you.” From Saine’s perspective, in order for homeopathy to be wrong, EVERYTHING has to be wrong, and the human mind, as we know, will not let that shift take place easily. Our world views are held in place by tenacious and deep-rooted self-referential social and neurological systems. To rise above such a world view and step into the clear harsh light of what’s “really real,” is an almost super-human act, because what such a person is leaving behind is not just a belief in this or that ridiculous (to us) idea but rather an entire life. What skeptics are trying to do in a debate such as this, in essence, is deprogram their opponents, and that ain’t gonna happen. Perhaps the whole debate process in this regard, when it comes to the skeptical vs believing communities, is a little self-referential and even narcissistic on the part of skeptics as well. It amounts to an ad hominem war on an entire class of people; namely, those who haven’t been fortunate enough to be born into or (yet) grow into a life — and, importantly, a livelihood — based on something that is really real. [Imagine for a moment that someone tried to convince you that YOUR job, your salary, your 401K, the support of your family, your kids' future, and the years and decades you've invested in it was all founded on a fallacy, and now you had to let it go, and GOOD LUCK to you out there in the job market. How would you react? Would you perhaps hang on a little to your biases, in spite of the lack of sufficient properly double blinded studies?]

    The debate has little to do with the people themselves, and this supposed “believing class” of humans we should be vigorously resisting does not actually exist. We are all born with believing minds, and as “pattern seeking primates” we will seek to make sense of our given reality as best we can. Casting the present situation in an “us vs. them” light serves to promote the success and popularity of various skeptics, bolstering their self-image, positioning their books and talks and podcasts and media appearances, etc., and, importantly, creates a sense of community. Skeptics have the advantage of being correct, which is awesome. But there’s a danger there…communities are tricky when it comes to the truth, because once we are inside them it is difficult to detect our own biases, as everyone here knows all too well. To rely even partially on the insider/outsider paradigm to inform our self-image is, in my opinion, to fall somewhere short of the clear, harsh truth and a more complete understanding of what’s truly, really real.

    That being said, I sure wish I could make it to Nexus this year. I really want to meet you and become part of this group.

  10. jt512 says:

    If you liked plausibility bias, you’ll positively love rationality bias:

    “The evolution story leaves the authority of reason in a much weaker position,” [Thomas Nagel] writes. Neo-Darwinism tells us that we have the power of reason because reason was adaptive; it must have helped us survive, back in the day. Yet reason often conflicts with our intuition or our emotion—capacities that must also have been adaptive and essential for survival. Why should we “privilege” one capacity over another when reason and intuition conflict? On its own terms, the scheme of neo-Darwinism gives us no standard by which we should choose one adaptive capacity over the other. And yet neo-Darwinists insist we embrace neo-Darwinism because it conforms to our reason, even though it runs against our intuition. Their defense of reason is unreasonable.

  11. cervantes says:

    Oh come on. The reason for privileging reason over intuition is because we can do tests to find out the truth. People had an intuition that heavy objects fall faster than light ones, but we can do experiments that prove this isn’t true once we account for air resistance. People had an intuition that the earth was at the center of the universe and all the heavenly bodies revolved around it, but that is indisputably false. Sure, intuition is adaptive — there isn’t always time or means to do rigorous investigation. But once we do have both, intuition often turns out to mislead. This is true in the exact case of evolution. Living things appear to us, intuitively, to have been designed, but once we do rigorous investigation of the world, we find a different, far more compelling, explanation.

  12. jt512 says:

    Steve Novella wrote:

    In short – statistical significance is about statistical noise in an unbiased random system. Signal to noise ratio (clinical significance in medicine) is about the probability that the results might be due to a biased system – to systematic bias.

    My favorite statistically significant finding of all time is the one by Bösch (2006) in their meta-analysis of 380 experiments to determine whether the human mind can influence the output of computer random number generators (RNGs). If there were no effect then the RNG should output 1s and 0s in equal proportion in the long run. Bösch (2006) found that, in fact, subjects actually did statistically significantly worse than chance. They’re result…wait for it… 0.499997, p=.0001. That finding was from their fixed-effects model. Then, as if to intentionally refute their own results, they reran the analysis using a random-effects model and got 0.500035, p=.0068, a statistically significant result in the opposite direction!

    Both models cannot be right; at least one of them has to be introducing a bias. The model-dependency of these results illustrate what you’re talking about: that small effect sizes can be the result of hidden biases in the execution or the analysis of an experiment, statistical significance not withstanding.

    The authors concluded that an effect of human intentionality on random number generators is “not proven.” And, of course, “further experiments will be conducted.” The verdict of “not proven,” reveals a deficiency of standard hypothesis testing, namely, that you can’t accumulate evidence in favor of the null hypothesis. Thus a study can have only two outcomes: a finding in favor of the research (ie, alternative) hypothesis or it can be inconclusive. A Bayesian analysis, in contrast, allows for a third possibility: The null hypothesis is upheld; no further research is required.

    Bösch (2006) also contains my all-time favorite exclusion criterion: “We also excluded [studies] that used animals or plants as participants.”

    Source: Bösch (2006). Examining Psychokinesis: The Interaction of Human Intention With Random Number Generators—A Meta-Analysis. Psycho Bull.* 32 (4), 497–523.

    *I just couldn’t resist.

  13. cervantes says:

    Or, to make a more general point, Gaussian statistics are really uninterpretable without information about prior probability; a p value can be evidence that affects our judgment about the likelihood of a conclusion, but that is not equal to 1 – p. You have to apply Bayes theorem. If the prior probability is zero, it’s still zero, no matter what you think you see in your trial. If it’s very small, it might get a little bit bigger, but it’s still small.

  14. estockly says:

    Dr. Novella,

    >>Defenders of science-based medicine are often confronted with the question (challenged, really): what would it take to convince you that “my sacred cow treatment” works? The challenge contains a thinly veiled accusation — no amount of evidence would convince you because you are a nasty skeptic.

    When I ask that question (as I am about to) I am not accusing you of being a nasty skeptic. I see it no different than when you ask if a claim is falsifiable.

    In a sense, isn’t homeopathy the low-hanging fruit? There is no science to support it and very little plausible aspects to study. It’s an excellent illustration of the point you’re making, but there is no real scientific dispute about it’s effectiveness.

    So, what would it take to convince you that a LCHF diet (my “sacred cow”) is a healthier and more effective weight loss strategy than the strategies based on calories in/calories out (eating less and exercising more).

    By healthier I mean improving risk factors for chronic disease; by more effective I mean lose more weight by metabolizing more stored fat while preserving more (or building more) lean tissue.

    The studies linked to on this page seem to fit your criteria:

    http://www.dietdoctor.com/science

    What more will it take to falsify your theory of macronutrients, diet and nutrition?

    ES

  15. Earthman says:

    Looking at a linked paper from the Homoeopathic Research Institute Newsletter, I find a weak argument that lacks rigour or structure. However there is a testable statement with a reference:-

    “The finding that homeopathic dilutions of histamine inhibit basophil degranulation has been verified repeatedly by different scientific teams.13″

    13. Endler PC, et al. Repetitions of fundamental research models for homeopathically prepared dilutions
    beyond 10-23: a bibliometric study. Homeopathy 2010;99:25-36

    I wonder if someone could have a look at this and see if there is any substance to this assertion.

    Thanks

  16. cervantes says:

    “Bibliometrics” is not a method for testing the validity of conclusions. It means tracing citations, basically, to see who is citing what literature. It’s a way of gauging the impact of research on a field or the linkages among various research enterprises. I haven’t read the paper but I’m guessing it ends up concluding that there are multiple studies in quack medicine publications that cite each other in a big circle jerk and end up making this preposterous claim.

    The way to test a proposition by combining evidence from multiple studies is called meta-analysis. This takes into account the quality of studies as well as combining their results. Obviously, that’s not what the citation did.

  17. windriven says:

    @Earthman

    Further to cervantes comment, the line between bibliometrics and an echo chamber can be vanishingly small. This is not to discount bibliometrics out of hand. But like meta analyses they require exceptional care if they are to have value.

    I have not read Endler’s paper either, nor am I likely to do so as it appeared in Homeopathy. One must consider the source when assigning weight to data.

  18. Scott says:

    Signal to noise ratio (clinical significance in medicine) is about the probability that the results might be due to a biased system – to systematic bias.

    Clinical significance isn’t that great a surrogate for systematic uncertainty. Unfortunately, it’s about the only one available given that medical research doesn’t consider systematic uncertainty in any kind of rigorous way.

  19. daedalus2u says:

    The way I look at intuition and reason is that “reason” is algorithmic, it allows you to manipulate facts using the algorithms of logic. Because those are all done explicitly, the train of logic can always be examined and verified. Is every “fact” correct? Is every logical operation valid? Then the conclusion follows from the premises.

    But sometimes you don’t have enough facts, or you don’t have time time to tie what facts you do have together in a logical structure. That is where intuition comes in. I see “intuition” as a non-algorithmic method to arrive at answers.

    An example of intuition is the estimation of quantity. Many organisms can estimate quantity, and can choose between more numerous and less numerous outcomes. They can do this without being able to count. Counting is an algorithm. If properly executed it will always give the correct answer, but there is a lot of computational overhead to run the counting algorithm. You are trading off degree of accuracy for speed. It is easy to tell “by eye” the difference between 10,000 items and 1,000 items without counting them both. It is more difficult to tell between 5,545 and 5,560 items.

    Intuition usually doesn’t give quantitative answers, it gives you a “gut feeling”.

    You can train yourself to have better intuition. When ever you have an intuition about something that turns out to be not correct, make a note of that and modify future thinking in those areas.

    To respond to cervantes, people use reason to get the answer that most closely corresponds with reality, they use intuition to get the answer that most closely corresponds with what they want. If you want your biases to be upheld, don’t use reason to analyze them.

    Most things don’t actually fit a Gaussian distribution, they fit a power-law distribution, where most of the “stuff going on” is out in the tails that are too infrequent to find in reasonable sized experiments. These are the “Black Swans” of Nassim Nicholas Taleb.

    https://en.wikipedia.org/wiki/Black_swan_theory

    Black Swans happen because the underlying dynamics are chaotic, that is comprised of multiple, coupled, non-linear systems. Such systems are inherently unpredictable because they are differentially sensitive to initial conditions.

  20. jt512 says:

    daedalus2u wrote:

    Most things don’t actually fit a Gaussian distribution, they fit a power-law distribution, where most of the “stuff going on” is out in the tails that are too infrequent to find in reasonable sized experiments.

    Most “things” fit a power-law distribution? How did you arrive at that conclusion, by counting up all the “things” in the world and checking their distributions, or by intuition?

  21. Lytrigian says:

    @Steve — On the one hand, I agree with you. Yes, the ideas and the thought processes must be resisted. When more people use more rationality in making important decisions, all society is better off.

    However, the people promoting pseudoscience and other irrationality *must* be resisted. They are harming others who buy into the world view, ideas, and products they are selling. When a cancer patient seeks out fake modalities like homeopathy or quacks like Burzynski because practitioners and shills hold out false hope that no honest real doctor can offer, they are as much as killing themselves. When an asthma patient seeks out acupuncture because it makes him feel better, even though his lung function afterward is still objectively impaired, he is putting himself at needless risk. The harms in all these sorts of treatments are well-documented. The situation is too serious to be gentle.

    Personally, I couldn’t care less if a charlatan like Saine is convinced or not. I care deeply about convincing his potential victims.

  22. Steve C says:

    @Lytrigian: point well taken. I agree. I was just suggesting that the strategy of trying to change/dissuade the people who are least likely to be changed (the charlatans/deluded practitioners themselves) is perhaps not the most rational approach.

  23. Lytrigian says:

    True, but in a public debate such as Dr. Novella is talking about, you’re not really trying to convince the other side. That pretty much never happens. Your target is the audience.

  24. Jan Willem Nienhuys says:

    Novella on http://www.sciencebasedmedicine.org/index.php/evidence-thresholds/#comment-117773

    P-value, in fact, is the probability of the results being what they are or greater assuming the null-hypothesis

    Almost correct. It is the computed probability for such results are better etc. You see, when you throw a die, and a six shows up, then that event “six on top” has happened, so it has probability 1. But you still can say that generally throwing an unbiased die will yield in 1/6 of the cases a six.

    Before this quoted phrase dr. Novella writes

    Statistical significance is used to determine if the results are likely to be due to random chance

    No. No. Noo. There is no way a single experiment can determine whether something is due to random chance. For that you have to investigate in depth the mechanism that produces the event. After you have established that your die is honest and after you have seen the manner in which is thrown, you may confidently say that the the outcome (whatever it is) is due to chance. If you win in a large legal lottery, you experience an event that is due to random chance (even if your feeling naturally tells you some higher power may have been rigging the result in favor of you).

    What you can do, however, is assume that random chance is operating and then calculate the probability for a certain type of event. It has become a tradition (in medicine) to argue then as follows.

    ‘if something with a computed chance of 1 in 20 really happens, we may call it a miracle. In case of miracles the Humean principle applies, namely that it is far more probable that there is some explanation, such as fraud, selective reporting, some kind of error or maybe maybe some kind of real effect.’

    I don’t know why this 0.05 threshold is so ridiculously low. If you would risk losing a million dollar on the outcome of a test (with no clear benefit if the test came out in support of the null hypothesis) you wouldn’t be so cavalier.

  25. daedalus2u says:

    jt512, everything that exhibits self-organizing criticality is exhibiting power-law behavior. That includes all living organisms. That includes sand dunes.

    https://en.wikipedia.org/wiki/Self-organized_criticality

    One of the most universal examples is 1/f noise.

  26. rork says:

    Missionary work, beware!
    “sufficiently powered studies that adequately define and control for all relevant variables”
    Factors are not relevant or irrelevant because I declare them to be so, unless I’m really pretentious.

  27. Jan Willem Nienhuys says:

    The ridiculous low threshold in homeopathy is inherent in the system. Remember, the idea is that if substance X produces phenomenon Y in a ‘healthy’ person, then a sick person that happens to also exhibit Y must be treated with X.

    So in the nineteenth century many volunteers took various X (usually 10 to power 60 times diluted stuff), unblinded of course and then wrote down everyting that happened to them – anything that was considered remarkable counted. Hahnemann himself had instructed that no prior condition should be taken into account. In this way thousands of pheneomena Y were collected for every single X. Many of these phenomena happened in only one test person. (In some cases the phenomena were taken from case reports of posionings in the literature, such as ‘the corpse was luminous in all its parts’ – symptom 1658 of Phosphorus in The Chronic Diseases vol II (original German edition 1835).

    Incidentally, this huge number of phenomena produced by highly diluted stuff is in my view the explanation for the old belief (of Hahnemann and his followers) that these highly diluted preparations have any effect at all. For the various kinds of magnetism (north pole, south pole, and ‘both poles’) there are almost a thousand phenomena! Most of the phenomena are very subjective.

    Some of these provings are repeated (reprovings), but never blinded of course. And then often people exhibited again very mnay symptoms. What happened then is that the expert looked in the symptom diaries, noted among the symptoms for X some of the known Y and declared the reproving successful.

    And if you think this is something of the past, even now you can find provings of diluted substances such as Jupiter light (light from the eyepiece of a telescopes that is pointed to Jupiter is shone into a bottle of alcohol, which is then properly diluted, and tested in the above manner.

    When sick people comes to the homeopath they have to talk about themselves (which is what many people like to do) and after a full hour of questioning the therapist has collected very many Y from the stories and asnwers to questions (are you afraid of thunderstorms? on what side do you usually sleep?) and then tries to find an X that has the most of these Y, or at least the most peculiar ones.

    The whole system has ‘low threshold of evidence’ built into it. Not only the remedies are highly diluted, the evidential quality of the whole system is diluted far past the vanishing point.

    The real problem is: how can academically trained MDs ever come to believe in this?

  28. Jeremy Praay says:

    (Thanks to Dr. Gorski for approving this post)

    Having been labeled a “denialist” in the past, and since I seem to agree so strongly with Dr. Novella’s most recent posts (the others being clinical decision making, I, II, and III), I have to wonder if these types of statements can be used by detractors in regards to what some may consider established science. (e.g. AIDS)

    In one area that I am intimately familiar with, shaken baby syndrome (an accepted diagnosis by most medical professionals), applying 1-4 is simply impossible, short of shaking babies to see what happened. Numbers 2 and 3 may be possible, but many of the published results that I have read seem to suffer from incredible problems of circular logic: the most atrocious, in my opinion, involves the significance of retinal hemorrhages. When retinal hemorrhages are the most prominent feature of the diagnosis, then stating that there is a statistically meaningful association between RH and SBS is meaningless, even when discounting all of the other known causes of RH in a “rule-out” diagnosis. i.e. It’s not A, B, C, or D, therefore it must be SBS, because RH was detected.

    Clearly, I believe the threshold has not been met, but I accept that I may also be wrong. I am truly puzzled that this diagnosis has not been attacked by the skeptic community. Dr. Caffey called SBS an “extraordinary diagnostic contradiction” and yet if there were ever extraordinary evidence to support the extraordinary claim, I have yet to find it.

    As CAM gains more prominence in medical schools, and the old skeptics die off, I see this as a harbinger of things to come. Perhaps in 20-30 years, skeptics of CAM will be in the small minority. But that will not mean that they are wrong. Will it?

  29. jt512 says:

    daedalus2u wrote:

    jt512, everything that exhibits self-organizing criticality is exhibiting power-law behavior. That includes all living organisms. That includes sand dunes.

    My objection was to your overreaching claim that “most things” have this property. In my 20 years as a statistician, I’ve examined the distributions a lot of “things,” and the overwhelming majority of them could not possibly have a power-law distribution.

  30. Jan Willem Nienhuys says:

    simply impossible, short of shaking babies to see what happened

    is it not possible to do animal experiments?

  31. Jeremy Praay says:

    @Jan: “is it not possible to do animal experiments?”

    These are the only two that I am aware of, at least recently.

    http://www.ncbi.nlm.nih.gov/pubmed/20435592

    http://www.ncbi.nlm.nih.gov/pubmed/22705132

  32. windriven says:

    @JWN

    Which animals would you shake to simulate shaken baby syndrome? That is to ask, which animal model would be an appropriate analogue to human infants?

  33. daedalus2u says:

    jt512, I don’t dispute your experience. I don’t even dispute that most things that statisticians calculate the dsitributions of are not power-law distributions. However, once you get more than a few non-linear coupled variables you get chaos. Heart rate is a classic example. Heart rate in healthy people is not regular, it is chaotic. Heart rate in sick people is more regular.

  34. jt512 says:

    daedalus2u, I only dispute your claim that “most things” follow a power-law distribution, an assertion that have not and cannot substantiate.

  35. I’d be interested to hear your take on this study that claims that most published research is false. It’s a touch beyond be to parse due to technical language.

    http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124

  36. geo says:

    Would these standards not be almost impossible for talking or behavioural therapies to meet? Certainly, blinding (at least of the patient) is practically very difficult for many interventions, as is developing reliable outcomes measures for non-blinded trials. Should these then be seen as CAM? Should we lower some standards, but require more demanding improvements before considering them to be evidence based treatments?

  37. jt512 says:

    @Logan: Steve has commented on that article before; I think it’s fair to say he’s a big fan of it. My take on the article is that a lot of the published research is false, but that Ioannidis stretched the definition of “false” to include some common, but innocuous faux pas, and didn’t actually prove that most research is false, although he got a catchy headline by saying so.

  38. daedalus2u says:

    My comment that “most things” occur in a power-law distribution was not meant to be taken literally; even though it literally is correct. The normal distribution

    https://en.wikipedia.org/wiki/Normal_distribution

    is actually quite rare. It is used so much because is is easy to use, and if you are trying to find an average it is convenient. If you want to consider things that are additive, it is ok. It you want to consider something more complicated than things that are averages or additive it isn’t that useful. Most things are more complicated than just averages.

    Another name for power-law distributions is the Pareto Distribution.

    https://en.wikipedia.org/wiki/Pareto_distribution

    and also Zipf’s law.

    https://en.wikipedia.org/wiki/Zipf%27s_law

    http://arxiv.org/pdf/cond-mat/0412004

    Things that are characterized by multiple-non-linear coupled interactions occur in a power-law distributions. Gravity is a non-linear interaction. Things characterized by gravity (i.e. gravitationally bound objects) should exhibit a power-law distribution, for example:

    Stellar mass:

    http://mnras.oxfordjournals.org/content/398/4/2177.abstract

    Asteroid mass.

    http://www.sciencedirect.com/science/article/pii/S0019103504003811

    Things that exhibit fractal size; that is sizes that are self-similar should also exhibit power-law size distributions, for example crystals such as snowflakes.

    http://sciencelinks.jp/j-east/article/200017/000020001700A0643786.php

    Grains produced through attrition by glaciers:

    http://rose.bristol.ac.uk/bitstream/handle/1983/1167/s17.pdf?sequence=2

    Interstellar grains produced through collisions.

    http://adsabs.harvard.edu/full/1996ApJ…469..740J

    Raindrop sizes

    http://journals.ametsoc.org/doi/abs/10.1175/1520-0469%281948%29005%3C0165:TDORWS%3E2.0.CO;2

    The most numerous object is very likely to be interstellar dust grains. If they fit a power-law, then “most things” fit a power-law distribution.

    The point I was trying to make (and was inarticulate in how I expressed it) is that if you don’t understand the underlying physics (which is usually true about medical data) it can be difficult to pick the correct mathematical correlations to use to evaluate the data. Using the wrong distributions and/or the wrong analysis can lead to problems, like the US housing bubble. The models they were using didn’t allow for housing prices to fall, so not surprisingly those models never predicted that the housing bubble could collapse when housing prices did fall. When these models are on spread sheets that can not be checked, the results can be disastrous.

    http://baselinescenario.com/2013/02/09/the-importance-of-excel/

    Very often data from natural sources is very sparse, so it can be difficult to know what type of statistical model to use.

    Choosing the wrong model is extremely common. For example the model that has been chosen for most types of exposure data is the linear-no-threshold (LNT) model. There is no data that requires a LNT-type model because in fact, the LNT model is wrong and does not actually fit any exposures that are important. The correct model is hormesis, but applying hormesis correctly is quite difficult. It is much easier to apply the LNT-type model incorrectly, so that is what is done.

    The LNT model vs hormesis is a good example for this post on evidence thresholds. Many people accept the LNT model and reject the hormesis model, even though the LNT model is wrong. This is an example of an acceptance made in the absence of evidence.

  39. jt512 says:

    My comment that “most things” occur in a power-law distribution was not meant to be taken literally; even though it literally is correct.

    Well, that’s an argument you don’t often see.

    You go on to name a handful of “things” that are supposedly distributed according to a power-law distribution, where you define “things” as the collection of “things” that have that distribution. You then try and pull a fast one by changing the definition of “things” you’ve been using all along to the individual “things” that are themselves so distributed. Nice try (not really). Treating these “things” correctly as a single “thing,” brings the total number of power-law-distributed “things” that you’ve enumerated to support your claim that most “things” follow a power law distribution to a handful + 1.

    You are apparently unable to admit that it is impossible in principle to say whether most “things” follow a power-law distribution. But here’s the real kicker: for many practical purposes it doesn’t matter, because regardless of the underlying distribution of the phenomenon, by the central limit theorem, its sample mean follows an approximate normal distribution; so even if the underlying phenomenon follows a power-law distribution, unless we’re specifically interested in the behavior of the individuals in the tail of the distribution, the normal distribution is generally the mathematically correct distribution to use.

  40. daedalus2u says:

    The context of the discussion was evidence thresholds and prior plausibility and how to update that using Bayesian methods. If you are looking for a single average, then yes, the normal distribution is a useful technique and can be used to converge on that single average.

    However the usual application of evidence thresholds is not to get an average, but to answer the relatively binary question, “is it true”, or “does it work”? The answer to the question “does homeopathy work” is not to be found in averages.

    If you want to get super pedantic, sure, I will admit that the term “most things” is poorly defined such that saying “most things” are X is also a poorly defined statement. The usual approach for skeptics when encountering poorly defined terms is try to improve definitions such that concepts are mutually understood. If the term “most things” is taken to mean “the most numerous solid objects in the Universe”, that would be grains of interstellar dust, which do follow a power-law distribution.

    I agree, the central limit theorem is useful, if what you are looking at has and average or a central limit. If you don’t know what you are looking for, or don’t know that it has a central limit, it can be problematic to assume that it does. The assumption of a central limit is what leads to the fallacy of false balance, that no matter what is at the extremes, the “true” answer is somewhere in the middle.

  41. S_Dave says:

    Re: Gorski

    Are you aware that there are people out there who suffer extensively under the hands of allopathic medicine, both as a result of their healing efforts and as a result of the lack of care? Consider the lame diagnosis of delusional parasitosis.

    There are many people who suffer from unusual and rarely diagnosed skin afflictions caused by one parasitic organism or another. They go from one doctor to another seeking help. The only help that the medical profession will offer is to call them delusional and place them on some anti-psychotic to calm them down, not even realizing that the anti-psychotic actually calms down the parasites too. Note that I say calm down, they are still there. Many doctors do not even bother to examine the patient to look for a real cause. In many of these cases, there is either a protozoal presence or a fungal presence, but in others, there are various forms of mites that the medical professional over looks or just does not find. Sadly, these people have a real health issue and the doctors in general just do not care enough to look. There are many other examples of why people turn to naturopathic healers.

    Energy medicine (reiki, therapeutic touch, etc.)

    I will tell you without a doubt, there is nothing more healing than touch when it is needed. There have been many studies done on infants who are deprived of touch and they develop numerous afflictions. The effects of touch can be observed with an EEG as well as with an EKG and with blood pressure monitoring. Heart rate, blood pressure, and brain activity influence every part of our bodies, including the activity of the immune system.

    Claims based on non-existent anatomical structures (reflexology, iridology, acupuncture, chiropractic subluxations)

    I had visible subluxations that were significantly improved by chiropractic. I have the before and after x-rays to prove it. If anyone really thinks that having a pinched nerve does not cause physiological problems than they really need to go back and study and learn. Even Iridology has a basis but I do agree that there are significant variables that have not been mapped yet. The presence of acupuncture
    points has been proven and it has been shown that there are nerve ganglia at these locations. Read “Body Electric” by Dr. Robert O. Becker, M.D.

    Non-existent physiologic functions (craniosacral rhythms, movement of bones in neurocranial restructuring, detoxification with coffee enemas or foot baths, dissolving tumors with orally administered pancreatic enzymes)

    If anyone thinks that having pressure exerted on the brain by a misaligned cranial plate does nothing, there is some fine ocean front real estate for them in Montana.

    The same for foot baths. It has been known for a long time that the feet are ion exchange organs and that people who frequently walk on the beach barefoot have lower levels of metalic toxins in their bodies. I agree that some practitioners will add things to the water to make it look more impressive.

    Try this: wash your feet until you think that they are perfectly clean. Take 1 quart of distilled water and add 1 tablespoon full of sea salt ( not table salt ) and 1 teaspoon of Epsom salt and dissolve completely. Get a roll of fresh clean surgical gauze with no contamination and soak it in this solution. Wrap your feet completely in this and place them in a plastic bag for a couple of hours. Keep some of the gauze aside as a control place this in a separate plastic bag. When you remove the gauze from your feet, you will often immediately see the difference, but even if not, sen both the gauze from the feet and the control gauze off for a metal analysis. You will find that there is a difference. The body will absorb sodium, calcium, magnesium, and some other metals while at the same time, excreting very minute amounts lead, mercury, cadmium, and some other metals.

    I know people who started doing coffee enemas back in the 1970′s. At 90 years of age, they are healthier than most people are at 70 years old. If you do not ‘believe’ in coffee enemas, that is fine with me. Your loss.

    Claims violating multiple laws of physics (homeopathy)

    I agree with this, anything that violates any law of physics is not probable.

    However, things such as reflexology, iridology, acupuncture, chiropractic subluxations, chelation, therapeutic touch, herbal remedies, and even zappers have a scientific basis and do not violate the laws of physics.

    http://www.healthcarereform4people.com offers some insight as to what causes us to be unhealthy.

  42. windriven says:

    @ S_Dave

    “I will tell you without a doubt, there is nothing more healing than touch when it is needed.”

    Yes, during cardiac arrest I would agree that the touch of someone skilled in CPR on one’s sternum would be, if not healing in the strictest sense, at least wildly appropriate.

    “However, things such as reflexology, iridology, acupuncture, chiropractic subluxations, chelation, therapeutic touch, herbal remedies, and even zappers have a scientific basis and do not violate the laws of physics.”

    This statement exposes you as beyond delusional. You might get by on a technicality on a few of these but each and every one would fail if we substituted ‘science’ for ‘physics.’ I include herbal remedies in this blanket because pharmacognosy is science and tincture of newt is … not.

    My sincere hope is that you will treat your every complaint with whichever bit of quackery you think appropriate and that you avoid the medicine that you call allopathic in every event.

  43. WilliamLawrenceUtridge says:

    Are you aware that there are people out there who suffer extensively under the hands of allopathic medicine, both as a result of their healing efforts and as a result of the lack of care? Consider the lame diagnosis of delusional parasitosis…There are many other examples of why people turn to naturopathic healers.

    That seems like a reason to improve real medicine, not embrace untestable or falsified claims. More specifically, that seems like an argument for improving how we deal with mental illness, the red-headed stepchild of medicine. Mental illness is orders of magnitude more difficult to study and treat because you aren’t dealing with simple one-to-one correlations (infection causes symptoms, antibiotics treat infection, infection goes away, no more symptoms).

    I’m not sure how people alleging they have Morgellon’s disease are better off by pretending there really are insects burrowing under their skin or somehow their body is exuding polyester. Supporting a delusion probably won’t help them stop scratching. It may make them feel better, but ultimately their health will probably suffer.

    If naturopathic healers genuinely help (in the form of getting rid of the parasite), then their methods should be brought into real medicine so more people can be helped and the actual causes treated. If naturopathic healers are doing little more than subjectively improving symptoms without changing the cause, if they are essentially providing a form of medicine-focused counseling, then what they are doing is taking money for spurious reasons, which strikes me as unethical. And that assumes that they don’t produce actual direct harms through missed diagnoses, spurious criticisms of vaccination or not-so-harmless treatments like high-dose vitamins or poisonous herbs.

    I will tell you without a doubt, there is nothing more healing than touch when it is needed. There have been many studies done on infants who are deprived of touch and they develop numerous afflictions. The effects of touch can be observed with an EEG as well as with an EKG and with blood pressure monitoring. Heart rate, blood pressure, and brain activity influence every part of our bodies, including the activity of the immune system.

    First, therapeutic touch doesn’t involve actual touch. Second, not everyone is a baby. Third, if touch was so effective, we wouldn’t have seen massive gains in longevity and infant survival after the advent of modern medicine (we had touch before we had vaccines and antibiotics). Touch may be one important component of treatment, but if you disconnect that respirator, a premature infant will die for certain no matter how much you hug it. Plus, I don’t think medical professionals advocate parents should not touch their children. Finally, I’d love to see a citation for this, I’ve seen this claim before and I’d be intrigued to see what the evidence base looks like.

    I had visible subluxations that were significantly improved by chiropractic. I have the before and after x-rays to prove it. If anyone really thinks that having a pinched nerve does not cause physiological problems than they really need to go back and study and learn.

    The problem isn’t with visible subluxations, it’s with invisible subluxations that some chiropractors claim cause completely unrelated diseases like cancer and diabetes. Actual joint manipulation is a recognized medical treatment in which chiropractors are redundant to physiotherapists and osteopaths. There is no question a pinched nerve is a real, painful condition. Chiropractic has about as much to do with pinched nerves as the Iraq war has to do with yellowcake uranium. It’s good press, it’s just not based on fact.

    Even Iridology has a basis but I do agree that there are significant variables that have not been mapped yet. The presence of acupuncture

    No it doesn’t. There are medical conditions that show up in the iris, but iridology goes beyond this to claim all medical conditions show up in the iris. This is simply, flatly wrong. You should read this piece by a former iridologist. There may be nerve ganglia in the eye, that mere fact doesn’t justify all of iridology. Science works by testing claims, allowing them to accumulate to justify a hypothesis. It doesn’t make a single observation and then claim an entire field of endeavor is justified.

    If anyone thinks that having pressure exerted on the brain by a misaligned cranial plate does nothing, there is some fine ocean front real estate for them in Montana.

    If you think that craniosacral therapy actually moves cranial plates, I have a bridge to sell you in Brooklyn. Again, yellowcake uranium and inappropriate leaps.

    The same for foot baths. It has been known for a long time that the feet are ion exchange organs and that people who frequently walk on the beach barefoot have lower levels of metalic toxins in their bodies. I agree that some practitioners will add things to the water to make it look more impressive.

    Again, trivially, this might be true. Whether this has a meaningful biological effect on a person’s quality or quantity of life is totally different. Foot baths feel nice. Have foot baths. Enjoy foot baths for what they are. Just don’t pretend they’re medicine for anything but corns and calluses.

    Oh, and citation needed for the claim about beach walkers.

    I know people who started doing coffee enemas back in the 1970′s. At 90 years of age, they are healthier than most people are at 70 years old. If you do not ‘believe’ in coffee enemas, that is fine with me. Your loss.

    Those people probably do a lot of things like exercise, maintain a reasonable body weight, eat lots of nutritious, unprocessed foods, spend time with family and friends and get enough sleep. Health behaviours cluster together, and teasing out which behaviour is responsible for which effect is difficult. This is what is meant by “correlation does not equal causation”. I eat breakfast every morning, and I have never been hit by a bus. This doesn’t mean eating breakfast prevents bus injuries.

    I agree with this, anything that violates any law of physics is not probable.

    However, things such as reflexology, iridology, acupuncture, chiropractic subluxations, chelation, therapeutic touch, herbal remedies, and even zappers have a scientific basis and do not violate the laws of physics.

    The selectivity of your skepticism is rather startling. Physics may not preclude most of those (therapeutic touch, certainly though – we know what energy is and we can measure it, and it is not involved in TT). The rest may not be precluded by physics (depending on what slippery claim you are specifically testing), but certainly by biology. Acupuncture is far better explained by psychology than by any specific biological effect. Herbs are just dirty drugs of uncertain potency (or completely ineffective). Zappers are nonsense – and probably violate the laws of physics. Certainly that reasonance nonsense does.

    While the website you link to does have some superficially good points (a lot actually, health care should be a basic right), it falls into quackery in the details. HFCS is just sugar. Statins are not poisons. Annual medical check-ups probably don’t do much good. Science can examine each of these beliefs empirically to determine if they are true or not. Science has already examined many of them and found them wanting. Science then discards them. SCAMsters and their credulous, exploited, conspiracy-theory-mongering patients never abandon a discredited hypothesis (particularly a lucrative one, making accusations of financial COI against Big Pharma both amusing and hypocritical).

  44. jt_512 says:

    daelelus2u wrote:

    However the usual application of evidence thresholds is not to get an average, but to answer the relatively binary question, “is it true”, or “does it work”? The answer to the question “does homeopathy work” is not to be found in averages.

    That binary question is usually answered based on averages; specifically, the difference between the average improvement in the treatment group and the control group in an RCT or a meta-analysis. I have never seen improvement in a symptom follow a power-law distribution.

    If the term “most things” is taken to mean “the most numerous solid objects in the Universe”, that would be grains of interstellar dust, which do follow a power-law distribution.

    Good for interstellar dust. The power-law distribution is irrelevant to determining whether homeopathy is effective.

  45. David Gorski says:

    However, things such as reflexology, iridology, acupuncture, chiropractic subluxations, chelation, therapeutic touch, herbal remedies, and even zappers have a scientific basis and do not violate the laws of physics.

    That’s one of the funniest things I’ve heard in a while. Therapeutic touch does actually violate the laws of physics. (Hint as well: There is no actual touching involved in therapeutic touch). Herbal remedies are nothing more than pharmacognosy rebranded as “alternative.” Look at it this way: If an herbal remedy works, it works because a substance in it is a drug (or substances in it). It’s a drug or drugs riddled with impurities and variable quality from lot to lot, but it’s a drug. Reflexology, iridology, acupuncture, and subluxations are based on nonexistent anatomic structures. The homunculi of reflexology and iridology do not exist. The meridians of acupuncture do not exist. (Oh, and redirecting the flow of qi does violate the laws of physics.) Subluxations do not exist. The parasites claimed to be “zapped” by the zapper are not the cause of the diseases that Hulda Clark said they were. Chelation has no biological plausibility and does not work.

    Maybe you have a different definition of “scientific basis” than I do.

  46. targeran1 says:

    Declaration of conflict of interests: I am an acupuncturist and a Naturopathic Physician. I am also a skeptic, though what I find myself most skeptical of at times is skepticism and so perhaps I should declare myself a self-styled meta-skeptic (with all due shame at using such a term I assure you).That said I also have a premedical undergraduate degree in Biological Sciences from Louisiana State University and in Arizona I was licensed as a primary care physician with full prescriptive right (less chemotherapeutics and certain antipsychotics). I couldn’t say anything about the USMLE exams on protocols and procedures performed in an emergency room but I am willing to take on any medical student, graduated or still in program, in the Step I basic sciences exams. That should be sufficient disclaimer to both sides of the issue in this blog post.
    When one speaks of truth as a scientist one is being sentimental. Truth is not what science provides; rather in is an eagerness for truth that defines what science is. Incorporating a vaster and more comprehensive field of facts is the practice of science. How quickly we have in our modern technological sprint forgotten the usefulness of Socrates’ revelation, “if I know anything it is that I know nothing.” For all of the history of science, be it primitive astrology or our own currently accepted version stemming back to the Greek modes of analysis, the one constant in scientific pursuit has been the obliteration of orthodox worldviews in the face of new facts coming to light. The facts we consider relevant today to our understanding of reality will not be immune to this mitigation.
    Skepticism about the medicine I practice is something I welcome. For one it pushes research in the field and broadens our understanding which in turn leads to refinement in theory and technique and thus greater safety and efficacy in clinical practice. Second, everything that is real is subject to criticism; another way of looking at it is that only tings which are real can be criticized, and this is basic didactics. Thirdly skepticism provides a rich arena for challenging dialogue and this just so happens to be a personal enjoyment of mine though it does make its contributions to improved patient care as well. Its no fun to play when the opponent does not take the game as seriously as you do.
    If anyone out there, allopathic or homeopathic or holistic or reductionist or empiricist or orthodox or conventional or naturopathic or whatever thinks that they can lay claim to the only correct interpretation of the facts then they are among the most delusional people on the planet now and throughout all of history. We can establish fact based only on what we can measure and what we can measure is limited by our technology which though it may fascinate us because it is the most recent to come on the scene is nonetheless very inadequate to establish anything close to a complete understanding of reality. Not only that but the facts we collect and establish are just that and we cannot measure what these facts mean, we can only interpret. There is no consistent, logical, inherent basis for determining things like ‘usefulness’ or ‘better’ or ‘worse;’ these are human values which are completely relative and not objective in any sense of the word.
    ‘Evidence threshold’ is a cleverly cloaked proselytizing by institutions first and most greatly concerned with their own perpetuation. It is easily seen to be so no matter which side of the coin you are looking from and neither institution discussed in the blog above, the allopathic nor the homeopathic, is innocent. This is an unavoidable situation in pursuits like modern scientific medical research which are so incredibly resource- dependent and -driven, both in material resources as well as specialization of knowledge and technique. Patients are constantly teaching the observant practitioner that contemplating what we have available to us as far as medicine is much more useful, revealing, and satisfying than concentrating on the fact that we have it. Other than that I find what has gone on the comment thread here most engaging and even entertaining. Thanks to everyone.

Comments are closed.