A new study published in the Archives of Pediatrics and Adolescent Medicine shows a positive correlation between counties in California, Oregon, and Washington with greater precipitation and a higher incidence of autism. While the results of this study are interesting, it needs to be put into proper context. Also of note, the authors had presented early results from this data previously.
Correlation is not Causation
This type of study is a correlational study, which means it asks whether or not there is a statistical correlation between two variables – in this case the rate of autism and the amount of precipitation. This type of data is extremely useful to medical science, but it has known limitations, which can be summarized by the statement that correlation is not causation.
I wrote previously on NeuroLogica blog about Morgellons disease. Both Peter Lipson and Wallace Sampson have also covered this interesting syndrome here on SBM. Briefly, sufferers of this dubious syndrome believe they have foreign material exuding from their skin, causing chronic itching and sores. The evidence suggests that in truth they suffer from something akin to delusional parasitosis – the false belief of foreign parasites in their skin, leading to chronic itching which causes the sores and also works clothing fibers into the skin, which are later exuded.
Morgellons, in short, is a fake disease, a false and somewhat far-fetched explanation for symptoms that have a much more prosaic, if undesired, explanation.
Those who believe they have Morgellons, however, are legitimately ill and are an extremely vulnerable population. They feel they have a serious and mysterious illness, and worse the medical profession does not understand their illness and so denies that it exists. This is a perfect setup for snake oil-peddling con-artists.
The science in science-based medicine includes all of science, but relies most heavily on the biomedical literature – published studies that collectively represent our scientific medical knowledge. The scientific basis of medicine is only as good as this body of knowledge and the manner in which it is interpreted and put into practice.
We often discuss on this blog how to evaluate individual studies- the need for blinding, randomization, the importance of study size to meaningful statistical analysis, and other features that distinguish a reliable study from a worthless one. This is important, but only half of the equation. We also at times discuss the medical literature as it relates to a specific medical question or set of related questions – does homeopathy work or are statins beneficial for cholesterol reduction, for example. This requires not only the ability to judge individual studies, but a higher order analysis of the overall pattern of evidence among all relevant studies. Failure to do this, by focusing only on individual studies, results in the failure to see the forest for the trees.
It is this higher order analysis that I wish to discuss in this entry.
When arguing against a specific scientific claim it is always desirable to be able to say that the claim violates an established law of science. Creationists attempt this with their argument that evolution violates the second law of thermodynamics (it doesn’t). The temptation is that such arguments are short and pithy, they seem conclusive, and they avoid the need to wade through a dense and complex set of scientific evidence and theories.
In the “diet wars” the first law of thermodynamics has been thrown around a lot. Up to now I have been aware of two camps defending their position with thermodynamic arguments. The first (and the one that I find most compelling) is the calorie in vs calorie out camp, that argues that the laws of thermodynamics apply to people too. This means that weight management must be a function of calories in (the total calories consumed by a person) – calories out (the total caloric expenditure, including metabolic processes, waste heat, exercise, and others). Thermodynamics must be obeyed and so if one wishes to lose weight they must burn more calories than they consume.
The Nobel Prize in Medicine was awarded this week to two French virologists, Françoise Barré-Sinoussi and Luc A. Montagnier, for discovering the AIDS causing virus, the Human Immunodeficiency Virus (HIV). They will share half the prize of 1.4 million dollars, the other half going to three Dr. Harald zur Hausen for discovering the human papilloma virus and its relation to cervical cancer.
The prize comes 25 years after Barré-Sinoussi and Montagnier, working at the Pasteur Institute in Paris, published their paper identifying what was later called HIV. The last quarter of a century has proven their discovery to be a triumph of science-based medicine. The Nobel committee is correct, in my opinion, in waiting such long periods of time before granting such recognition. It reflects that fact that, even in a fast-paced arena of science such as medicine, it takes time for the meticulous process of science to work itself out. It takes decades to garner the perspective necessary to tell the difference between a crucial breakthrough and a false lead.
Humans have the very odd ability to hold contradictory, even mutually exclusive, ideas in their brains at the same time. There are two basic processes at work to make this possible. The first is compartmentalization – the ideas are simply kept separate. They are trains on different tracks that never cross. We can switch from to the other, but they never crash into each other.
When contradictory ideas do come into conflict this causes what psychologists call “cognitive dissonance.” We then typically will relieve cognitive dissonance, which is an unpleasant state, through the second process – rationalization. We happily make up reasons why the two conflicting ideas actually don’t conflict at all. People are generally good at rationalization. It is a supreme intellectual irony that greater intelligence often leads to a greater ability to rationalize with both complexity and subtlety, and therefore a greater capacity to maintain contradictory beliefs.
In fact the demarcation between science and pseudoscience is often determined by the difference between sound scientific reasoning and sophisticated rationalization.
While cognitive dissonance refers to a process that takes place within a single mind, it is a good metaphor for the contradictory impulses of groups of people, like cultures or institutions. I could not help but to invoke this metaphor when reading two editorials published in the same day in the New York Times.
In yet another round of science by press release, a particularly unimpressive acupuncture study is making the rounds of the major news outlets proclaiming that acupuncture works. I guess that is a sort-of answer to my title question – why are so many scientifically worthless acupuncture studies being done?
Let’s take a look at this particular study to see why it is so weak. All I have to go on is the press release, since the study is not published. It was presented at a scientific meeting – which is legitimate, I just don’t have access to it. (The bar for publication in a peer-reviewed journal is much higher than presentation at a meeting, and there may, in fact, be changes to the text prior to publication.) But we can still say a great deal about this study from the information provided.
Bisphenol A (BPA) is a chemical used in the manufacture of hard plastics which can be found in a wide range of products, including baby bottles, plastic utensiles, and plastic food containers. It has been the focus of some controversy over its safety, and the resulting debate reveals much about how the current system deals with such issues.
The concern is that BPA can leech from plastic containers into the food or liquid it contains, and when consumed can have negative health effects. The debate is over how to interpret existing evidence about BPA safety, which gives conflicting results. Essentially it is a debate about how to weight different kinds of evidence, and where safety thresholds should be.
The Science of Toxins
Toxicity is always a function of dose. Anything is toxic at high enough dose, and safe at a low enough dose. Regulatory agencies concerned with protecting the public health, therefore, typically use scientific evidence to establish doses that are likely to cause toxicity in humans and then set safe levels of exposure significantly below that level to create a buffer of safety. But what kind of evidence is used?
The Center for Disease Control (CDC) conducts an ongoing telephone survey of medical problems and health care utilization – called the National Health Interview Survey (NHIS). They recently released their data from 2007. This is the first year for which they specifically broke out questions assessing the use of so-called complementary and alternative medicine (CAM).
CAM is a political/ideological entity, not a scientific one. It is an artificial category created for the purpose of promoting a diverse set of dubious, untested, or fraudulent health practices. It is an excellent example of the (successful) use of language as a propaganda tool.
The fundamental intellectual flaw of “CAM” as a concept is that it is made to include modalities that are extremely diverse, even mutually contradictory, under one umbrella. Very deliberately modalities which are scientific and mainstream, like the proper use of nutrition, are often included under the CAM umbrella by proponents in order to make it seem like CAM is a bigger phenomenon than it actually is, and as a wedge to open the door for the more pseudoscientific modalities.
Peer-review is a critical part of the functioning of the scientific community, of quality control, and the self corrective nature of science. But it is no panacea. It is helpful to understand what it is, and what it isn’t, its uses and abuses.
When the statement is made that research is “peer-reviewed” this is usually meant to refer to the fact that it has been published in a peer-reviewed journal. Different scientific disciplines have different mechanisms for determining which journals are legitimately peer-reviewed. In medicine the National Library of Medicine (NLM) has rules for peer-review and they decide on a case by case basis which journals get their stamp of approval. Such journals are then listed as peer-reviewed.