May 04 2012
I usually rely on the Secret. Every two weeks or so the Universe offers up some bit of wacky whimsey and I have a topic for an SBM blog entry. This week the Universe has failed me. Nothing has crossed my LCD so I have no studies to evaluate and I have been unusually busy at work preventing my browsing the Interwebs for material. But try telling that to the Managing Editor. I write half to amuse myself, half to learn about the topic, and half to clarify in my own mind the topics at hand (1). So this week is content free idle thoughts for my own benefit.
I have been reading 13 Things That Don’t Make Sense by Michael Brooks. The book concerns topics in science that are unexplained by the current understanding of the laws of the universe or contradict the dominant paradigm. Well, almost. His final topic is homeopathy, and it is the one topic whose conclusions, while qualified, belong on Failblog. The first chapter concerns dark matter and dark energy and how what we can see makes up only a small fraction of the content of the universe. The author discusses the history behind the discovery of dark matter and dark energy, and the theories that attempt to explain the cosmological measurements that demonstrate the dark existence. Besides new kinds of matter, physicists have hypothesized that the problem is not undiscovered matter but a lack of understanding of the nature of gravity. He discusses at length the Pioneer anomaly, the fact that the Pioneer space probes, launched in the 1970’s, are not following their projected trajectories and perhaps this was a manifestation of the dark matter/dark energy/modified gravity. Evidently it wasn’t, the perturbations are probably due to radiation pressures.
What is striking is the description of the approach to the measurements that led to the hypothesis of dark energy and matter and the Pioneer anomaly. The investigators took extreme care in looking for every possible error within known physical parameters rather than looking for new physics to explain the data. The lead investigator of the Pioneer anomaly said something to the effect, when you go into something looking for something new, that is what you will see.
I do not have an exact quote, since I am reading (2) the Audible book version, and the one problem with audio books is you cannot underline important quotes.
That is a striking characteristic of the evaluations of observations that contradict the known laws of the universe. First, the researchers were meticulous at looking into every possible reason for the anomalous results within known laws. And when they found anomalies, like the paths of stars circling galactic centers that suggesting the presence of dark matter, they carefully peat and repeat the measurements to confirm the findings before releasing the results.
It is the care with which physicist/astronomers and others in the ‘hard’ scientist evaluate the natural world that is impressive. They consistently rely first on the basic principals that have centuries of careful observation and experiment to validate their approximate truth. In the case of the Pioneer anomaly, there was no new physics to explain the trajectory, although it took 30 years to find the solution. In the case of stars orbiting the galactic center and the accelerating expansion of the universe, something unexplained was occurring. In both cases, a meticulous comparison to known reality was key to the understanding of the validity of the observed phenomena. They relied on basic principals.
The author also notes what happens when investigators evaluate phenomena with pre-existing ideas as to what they should find. I have always used the example of N-Rays, but Brooks uses Percival Lowell as an example. Lowell was an astronomer who was committed to finding a civilization of Mars, and as a consequence saw and mapped a complex canal system on the planet. The ability to see what you want to be there, rather than what is actually there, is one of the hallmarks of bad science, although in Lowell’s case he may have been inadvertently mapping the vessels of his own eye. I initially wanted to write an entire entry on the topic of scientific paradolia, but besides N-rays and Mars canals, I could not come up with similar examples. Freud? Perhaps.
The counter example, the disasters that result from ignoring basic principles, and of avoiding repetition and careful measurement was typified by Pons and Fleishman and the cold fusion debacle. There are phenomena not yet dreamt of in our philosophy, but if you are going to discover results that violate the basic laws of the universe, it had best be measured carefully and consistently reproduced. Cold fusion does not meet those criteria.
It is quite a contrast with the studies of SCAM modalities. Most SCAMs have no basis in known physics, chemistry, physiology, anatomy, etc. Acupuncture, homeopathy, reiki, chiropractic and their ilk are not based in reality but fantasy. It is the concept of prior plausibility, or what I would call a reality bias (3), that makes any positive findings more likely to error and bias. It is clear, particularly well delineated with acupuncture, that these modalities have no effect on objective endpoints and only minor effects on subjective endpoints. Increasingly well designed studies reveal decreasing effects until excellent studies show no effect. SCAMs are the medical N- rays and Mars canals. Yet in medicine, rather than noting that the studies are based in unreality and repeating careful studies that help tease out the biases and mistakes that make most SCAM trials appear to have effectiveness, they open Centers for integrative, alternative and complementary therapies.
It is odd, the blindness of the medical practice for rank nonsense. Brooks describes the opprobrium that Pons, Fleishman and others received for their trumpeting of imaginary breakthroughs, although I suspect it was not what they did, but how they did it, bypassing peer review in favor of a press release, that led to the pariah status they now apparently have. Researchers in quackery, rather than exile in France, start Clinics and Institutes. While the collapse of the careers of Pons and Fleishman et. al as described by Brooks seems excessive for the ‘crime,’ the only consequence in medicine for practicing magic is cash.
Outside of medicine careful research confirms or denies our understanding of reality. In medicine, at least with SCAMs to judge from the Bravewell report, reality is ignored.
The last two chapters of the book cover placebo and homeopathy. In the case of placebo, Brooks is a little sloppy at times at differentiating the difference between subjective and objective outcomes. In the example of homeopathy he focuses on a few anomalous studies that were never reproduced and a long, and interesting, description of mysteries of water, to wonder about the unknown properties of H2o that could validate homeopathy, all the while admitting homeopathy is, by current understanding, total nonsense. But as he consistently noted in the prior chapters, anomalous studies that can’t be reproduced and violate the basic laws of science turn out to be nonsense. Then he ignores the same lessons in his conclusions about homeopathy: based on a bit of popularity of the crowd (how can millions be wrong?) and the fact that since there other anomalies have led to insights, homeopathy may indeed have something to it. I doubt it.
Understanding often advances with anomalous studies that are reproducible and where more mundane explanations consistent with known physical laws are excluded. As Asimov noted, “The most exciting phrase to hear in science, the one that heralds new discoveries, is not “Eureka” but “That’s funny…” SCAMs remain funny, but not in the way Asimov suggested As they stand today, the anomalies of SCAM are no more than error and bias. If Ioannidis is correct about real medicine, and he is, then he is doubly correct for SCAM research. Instead of learning from past research, marginally positive results of poorly designed studies are seen as proof of efficacy, despite no prior plausibility for and the opportunities for error and bias are enormous. And then you charge the ill and vulnerable for the privilege of receiving your nonsense.
Astrology is not taught as a subset of astronomy for good reason. There is no alternative aviation, complementary engineering or integrative fire fighting. Medicine is different and it sometimes saddens me.
1) 3 out of 2 Americans do not understand statistics.
2) Yes, I am reading the book. Unless you say the blind are feeling the book.
3) Foreshadowing of SBM to come.
21 Responses to “Stop Making Sense”