Articles

Changing Your Mind

Why is my mind so clean and pure?  Because I am always changing it.
In medical school the old saying is that half of everything you learn will not be true in 10 years, the problem being they do not tell which half.
In medicine, the approach is, one hopes, that data leads to an opinion.  You have to be careful not to let opinion guide how you evaluate the data.  It is difficult to do, and I tell myself that my ego is not invested my interpretation of the data. I am not wrong, I am giving the best interpretation I can at the time. For years  I yammered on about how it made no sense to give a beta-lactam and a quinolone for sepsis until a retrospective study suggested benefit of the combination.  Bummer. Now when I talk to the housestaff about sepsis, I have to add a caveat about combination therapy.  It is why my motto is, only half jokingly,  ”Frequently in error, never in doubt”.
At what point do you start to change you mind?  Alter your message as a teacher?  Have new behavior?  Medicine is not all or nothing, black and white.  Changes are incremental, and opinions change slowly, especially if results of a new study contradict commonly held conclusions from prior investigations.
Nevertheless, I am in the process of changing my mind, and it hurts.  I feel like Mr. Gumby. (http://www.youtube.com/watch? v=IIlKiRPSNGA)
It is rare that there is one study that changes everything; medicine is not an Apple product.  Occasionally that there is a landmark  study that alters practice in such a dramatic way that there is a before and after.  As I write this I cannot think of a recent example in infectious diseases, but I am sure there is one.  The problem is that once practice changes, it seems as we have always done it that way.
For me, three is the magic number.  One study that goes against received wisdom warrants an ‘interesting, but give me more.”
Two studies, especially if using different methodologies with the same results gives and ‘well, two is interesting, but I can argue against it.”  However, with two studies the seed of doubt is planted, waiting to be watered with the water of further confirmation.  Yeah. Bad metaphor.
Three studies with different methodologies independently confirming new concepts?  Then I say, “I change my mind. My brain hurts.”
There are now three studies concerning the issue of efficacy of the flu vaccine in the elderly.  You might remember my discussion of the Atlantic article several months ago. In that entry I discussed two articles  that suggested the flu vaccine may be less effective in the elderly than the studies demonstrated. http://www.sciencebasedmedicine.org/?p=2495
The argument was that the elderly who received the influenza vaccine were healthier at baseline than those that didn’t receive the vaccine and the deaths during flu season was not due to the protection from the vaccine, but due to the fact that healthier people are less likely to die when they get ill. In part this was demonstrated by showing decreased deaths in vaccinated populations when influenza was not circulating.  If insomnia is a problem, you can go back and read my post.   To quote my favorite author, me, I said
“One, it is an outlier, and outliers need confirmation. The preponderance of all the literature suggests that influenza vaccine prevents disease and death. If you do not get flu, you cannot die from flu or flu related illnesses. When outliers are published, people read them, think, “huh, that’s interesting”, but there is going to have to be more than one contradictory study to change my practice. But if “study after study” shows mortality benefit, and one study does not, it is food for thought, but not necessarily the basis of changing practice. The results, above all, needs to be repeated by others… In medicine we tend to be conservative about changing practice unless there is a preponderance of data to suggest a change is reasonable. Except, of course, if our big pharma overlords take us to a good streak house.”
Now we have a third article, “Evidence of Bias in Studies of Influenza Vaccine Effectiveness in Elderly Patients” from the Journal of Infectious Diseases.
In the study they examined the records of the elderly in the Kaiser Health System, their vaccination records, and their risk of death.  And the results were interesting.
“The percentage of the population that was vaccinated varied with age. After age 65, influenza vaccination increased until age 78 in women and age 81 in men, then decreased with increasing age. Vaccination coverage also varied in a curvilinear fashion with risk score, increasing with risk score to a risk score percentile of ∼80%, then decreasing. In addition, as the predicted probability of death increased, vaccination coverage increased. Vaccination coverage was highest among members with a probability of death of 3%–7.5%. Those with a predicted probability of death in the coming year of 17.5% had a de- creasing likelihood of influenza vaccination”
They then looked at mortality when flu was not circulating.
“A change in the pattern of vaccination had a striking effect on mortality. For members > 75 years old who had been receiving influenza vaccinations in previous years, not receiving a seasonal influenza vaccination was strongly associated with mortality in the months ahead (Table 1). A person who had received an influenza vaccination every year in the previous 5 years had a more than double probability of death outside the influenza season if he or she missed a vaccination in the current year, compared with a person who was vaccinated as usual (odds ratio, 2.17; P < .001). On the other hand, if a person did not receive any seasonal influenza vaccination in the previous 5 years, then receipt of a vaccination in the current year was associated with a greater probability of death. “
If they had a history of flu vaccine for five years and missed it, the probability of death went up.
If they did not have a flu vaccine for five years and got one, the probability of death went up.
They suggest in the first case, the patients may have had an increase in their co-morbidities and as a result did not get the vaccine and died of underlying diseases. Their increased risk of death was from accumulating prior illnesses.
In the second case, people who were healthy and did not seek care subsequently developed diseases that lead them to a doctor who advised the vaccine.  Their increase risk of death was due to new illnesses.
Either way, the uptake of the flu vaccine is more complicated than I had suspected and makes interpretation of efficacy of the vaccine in prior studies harder to evaluate.  The table shows an unexpected relationship between age, risk of death and use of the flu vaccine.
table here
They say in the discussion
“We showed that, despite strong efforts to increase vaccination among the elderly population, vaccination is relatively low in the oldest and sickest portions of the population. Persons 65 years old with a 17.5% chance of death in the upcoming year are less likely to receive the influenza vaccine. Because persons who are most likely to die are less likely to receive the vaccine, vaccination appears to be associated with a much lower chance of dying; thus, the “effectiveness” of the vaccine is in great part due to the selection of healthier individuals for vaccination, rather than due to true effectiveness of the vaccine. Previous studies have argued that worsening health is associated with increasing vaccination. We found this to be a curvilinear relationship, in which increasing illness means increasing vaccination, up to a point, and then, as people come closer to the end of life, there is a decrease in vaccination coverage.”
They do not say the vaccine is not effective, but they suggest that there is a bias that may make the vaccine appear more effective in the elderly than it really is.  Reality is often more complex than one would think at the beginning.
After three studies I am reasonably convinced that efficacy of the flu vaccine in the elderly is potentially not as well understood as I had thought.
So do I think the flu vaccine is no longer useful in the elderly?  No.  I still think it is a reasonable intervention but it may not have the efficacy I would like.  But I have always known that, for a variety of reasons, the flu vaccine is not a great vaccine. But it is better than no vaccine. There are, as discussed in the earlier post on the vaccine, many lines of evidence to show that the flu vaccine has benefit; at issue is the degree of the benefit.  Perhaps what is needed is a better vaccine with adjuvants or multiple injections to get a better result in the elderly, who respond poorly to the vaccine.  Or perhaps it will be better to focus on increasing vaccination in those who care for or have contact with the elderly.  But when I talk to my patients and residents, when I get to part about flu vaccine efficacy, I will be a little more nuanced, use more qualifiers. I will tell them that the vaccine is like seat belts.  It does not prevent all death and injury, but if you had a choice, would you not choose to use seat belts?
In the end the data has to change the way I think about medicine, not matter how much it hurts.
Compare and contrast that with the anti-vaxers who have the belief that vaccines cause autism.  They look for data to support the pre-existing belief and ignore contrary data.  Opinion does not follow from data.
The most representative statement of their approach is on the 14 studies website where they say  ”“We gave this study our highest score because it appears to actually show that MMR contributes to higher autism rates.”
The key phrase in the whole site. Data that supports their position is good, data that does not is bad. What makes a study good is not its methodology or its rigor, or its reproducibility, or its biologic plausibility,  but if it supports vaccines casing autism.
Dr. Wakefield, as has been noted over the last week, had his MMR/autism paper withdrawn from Lancet not for bad science, but for dishonest science.  In medicine you can be wrong, but you cannot lie.  If the results of medical papers were shown to be fabrications, such as the papers of Scott S. Reuben, no one the medical field would defend the results.  Dr. Reuben, as you may remember http://www.sciencebasedmedicine.org/?p=408, was found to have fabricated multiple studies on the treatment of pain.  Nowhere can I find web sites defending his faked research.  No suggestions it was due to a conspiracy of big pharma to hide the truth. No assertions that he is still a physician of great renown.   He lied and is consigned to ignominy.   Physicians who used his papers as a basis of practice no longer do so, or so I would hope.
The response to Dr. Reuban is in striking contrast to the defense of Dr Wakefield, where bad research combined with unethical behavior, results in reactions like this
“It is our most sincere belief that Dr. Wakefield and parents of children with autism around the world are being subjected to a remarkable media campaign engineered by vaccine manufacturers reporting on the retraction of a paper published in The Lancet in 1998 by Dr. Wakefield and his colleagues.
The retraction from The Lancet was a response to a ruling from England’s General Medical Council, a kangaroo court where public health officials in the pocket of vaccine makers served as judge and jury. Dr. Wakefield strenuously denies all the findings of the GMC and plans a vigorous appeal.”
Opinions did not change when the Wakefield paper was demonstrated to be not just wrong but false, the researcher’s behavior unethical, and the study could not be reproduced using similar methodologies (http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0003140).  Instead, the defense of Dr. Wakefield became, well, like a Jim Carrey shtick. The Mask defends retracted autism research. Fire Marshall Bill on the medical literature.  Jenny and Jim’s defense does make more sense read as comic performance art.  Andy Kaufmann would have been proud.
I wonder if the more grounded in fiction an opinion is, the harder it is to change, the more difficult it is to admit error.  I have to admit I cannot wrap my head around the ability of people to deny reality.  It is the old Groucho line come to life, “Who are you going to believe, science or your lying eyes?”
So I will, I hope, keep changing my mind as new information come in.  It is what separates real health care providers from acupuncturists and homeopaths and naturopaths and anti-vaxers.  It is what some truly great minds admit to doing (http://www.edge.org/q2008/q08_index.html).  As one deeper thinker and better writer (http://www.emersoncentral.com/selfreliance.htm) than I said, kind of,
“The other terror that scares us from self-trust is our consistency; a reverence for our past act or word, because the eyes of others have no other data for computing our orbit than our past acts, and we are loath to disappoint them.
But why should you keep your head over your shoulder? Why drag about this corpse of your memory, lest you contradict somewhat you have stated in this or that public place? Suppose you should contradict yourself; what then? It seems to be a rule of wisdom never to rely on your memory alone, scarcely even in acts of pure memory, but to bring the past for judgment into the thousand-eyed present, and live ever in a new day. In your metaphysics you have denied personality to the Deity: yet when the devout motions of the soul come, yield to them heart and life, though they should clothe God with shape and color. Leave your theory, as Joseph his coat in the hand of the harlot, and flee.
A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines and anti-vaxers. With consistency a great soul has simply nothing to do. He may as well concern himself with his shadow on the wall. Speak what you think now in hard words, and to-morrow speak what to-morrow thinks in hard words again, though it contradict every thing you said to-day. — ‘Ah, so you shall be sure to be misunderstood.’ — Is it so bad, then, to be misunderstood?”

Why is my mind so clean and pure?  Because I am always changing it.

In medical school the old saying is that half of everything you learn will not be true in 10 years, the problem being they do not tell which half.

In medicine, the approach is, one hopes, that data leads to an opinion.  You have to be careful not to let opinion guide how you evaluate the data.  It is difficult to do, and I tell myself that my ego is not invested my interpretation of the data. I am not wrong, I am giving the best interpretation I can at the time. For years  I yammered on about how it made no sense to give a beta-lactam and a quinolone for sepsis until a retrospective study suggested benefit of the combination.  Bummer. Now when I talk to the housestaff about sepsis, I have to add a caveat about combination therapy.  It is why my motto is, only half jokingly,  ”Frequently in error, never in doubt”.

At what point do you start to change you mind?  Alter your message as a teacher?  Have new behavior?  Medicine is not all or nothing, black and white.  Changes are incremental, and opinions change slowly, especially if results of a new study contradict commonly held conclusions from prior investigations.

Nevertheless, I am in the process of changing my mind, and it hurts.  I feel like Mr. Gumby.

It is rare that there is one study that changes everything; medicine is not an Apple product.  Occasionally that there is a landmark  study that alters practice in such a dramatic way that there is a before and after.  As I write this I cannot think of a recent example in infectious diseases, but I am sure there is one.  The problem is that once practice changes, it seems as we have always done it that way.

For me, three is the magic number.  One study that goes against received wisdom warrants an ‘interesting, but give me more.’

Two studies, especially if using different methodologies with the same results gives and well, two is interesting, but I can argue against it.’ However, with two studies the seed of doubt is planted, waiting to be watered with the water of further confirmation.  Yeah. Bad metaphor.

Three studies with different methodologies independently confirming new concepts?  Then I say, ‘I change my mind. My brain hurts.’

There are now three studies concerning the issue of efficacy of the flu vaccine in the elderly.  You might remember my discussion of the Atlantic article several months ago. In that entry I discussed two articles  that suggested the flu vaccine may be less effective in the elderly than the studies demonstrated.

The argument was that the elderly who received the influenza vaccine were healthier at baseline than those that didn’t receive the vaccine and the deaths during flu season was not due to the protection from the vaccine, but due to the fact that healthier people are less likely to die when they get ill. In part this was demonstrated by showing decreased deaths in vaccinated populations when influenza was not circulating.  If insomnia is a problem, you can go back and read my post.   To quote my favorite author, me, I said

“One, it is an outlier, and outliers need confirmation. The preponderance of all the literature suggests that influenza vaccine prevents disease and death. If you do not get flu, you cannot die from flu or flu related illnesses. When outliers are published, people read them, think, “huh, that’s interesting”, but there is going to have to be more than one contradictory study to change my practice. But if “study after study” shows mortality benefit, and one study does not, it is food for thought, but not necessarily the basis of changing practice. The results, above all, needs to be repeated by others… In medicine we tend to be conservative about changing practice unless there is a preponderance of data to suggest a change is reasonable. Except, of course, if our big pharma overlords take us to a good streak steak house.”

Now we have a third article, “Evidence of Bias in Studies of Influenza Vaccine Effectiveness in Elderly Patients” from the Journal of Infectious Diseases.

In the study they examined the records of the elderly in the Kaiser Health System, their vaccination records, and their risk of death.  And the results were interesting.

“The percentage of the population that was vaccinated varied with age. After age 65, influenza vaccination increased until age 78 in women and age 81 in men, then decreased with increasing age. Vaccination coverage also varied in a curvilinear fashion with risk score, increasing with risk score to a risk score percentile of ∼80%, then decreasing. In addition, as the predicted probability of death increased, vaccination coverage increased. Vaccination coverage was highest among members with a probability of death of 3%–7.5%. Those with a predicted probability of death in the coming year of 17.5% had a decreasing likelihood of influenza vaccination”

They then looked at mortality when flu was not circulating.

“A change in the pattern of vaccination had a striking effect on mortality. For members > 75 years old who had been receiving influenza vaccinations in previous years, not receiving a seasonal influenza vaccination was strongly associated with mortality in the months ahead (Table 1). A person who had received an influenza vaccination every year in the previous 5 years had a more than double probability of death outside the influenza season if he or she missed a vaccination in the current year, compared with a person who was vaccinated as usual (odds ratio, 2.17; P < .001). On the other hand, if a person did not receive any seasonal influenza vaccination in the previous 5 years, then receipt of a vaccination in the current year was associated with a greater probability of death. “

If they had a history of flu vaccine for five years and missed it, the probability of death went up.

If they did not have a flu vaccine for five years and got one, the probability of death went up.

flu risk

They suggest in the first case, the patients may have had an increase in their co-morbidities and as a result did not get the vaccine and died of underlying diseases. Their increased risk of death was from accumulating prior illnesses.

In the second case, people who were healthy and did not seek care subsequently developed diseases that lead them to a doctor who advised the vaccine.  Their increase risk of death was due to new illnesses.

Either way, the uptake of the flu vaccine is more complicated than I had suspected and makes interpretation of efficacy of the vaccine in prior studies harder to evaluate.  The table shows an unexpected relationship between age, risk of death and use of the flu vaccine.

They say in the discussion

“We showed that, despite strong efforts to increase vaccination among the elderly population, vaccination is relatively low in the oldest and sickest portions of the population. Persons 65 years old with a 17.5% chance of death in the upcoming year are less likely to receive the influenza vaccine. Because persons who are most likely to die are less likely to receive the vaccine, vaccination appears to be associated with a much lower chance of dying; thus, the “effectiveness” of the vaccine is in great part due to the selection of healthier individuals for vaccination, rather than due to true effectiveness of the vaccine. Previous studies have argued that worsening health is associated with increasing vaccination. We found this to be a curvilinear relationship, in which increasing illness means increasing vaccination, up to a point, and then, as people come closer to the end of life, there is a decrease in vaccination coverage.”

They do not say the vaccine is not effective, but they suggest that there is a bias that may make the vaccine appear more effective in the elderly than it really is.  Reality is often more complex than one would think at the beginning.

After three studies I am reasonably convinced that efficacy of the flu vaccine in the elderly is potentially not as well understood as I had thought.

So do I think the flu vaccine is no longer useful in the elderly?  No.  I still think it is a reasonable intervention but it may not have the efficacy I would like.  But I have always known that, for a variety of reasons, the flu vaccine is not a great vaccine. But it is better than no vaccine. There are, as discussed in the earlier post on the vaccine, many lines of evidence to show that the flu vaccine has benefit; at issue is the degree of the benefit.  Perhaps what is needed is a better vaccine with adjuvants or multiple injections to get a better result in the elderly, who respond poorly to the vaccine.  Or perhaps it will be better to focus on increasing vaccination in those who care for or have contact with the elderly.  But when I talk to my patients and residents, when I get to part about flu vaccine efficacy, I will be a little more nuanced, use more qualifiers. I will tell them that the vaccine is like seat belts.  It does not prevent all death and injury, but if you had a choice, would you not choose to use seat belts?

In the end the data has to change the way I think about medicine, not matter how much it hurts.

Compare and contrast that with the anti-vaxers who have the belief that vaccines cause autism.  They look for data to support the pre-existing belief and ignore contrary data.  Opinion does not follow from data.

The most representative statement of their approach is on the 14 studies website where they say  ”“We gave this study our highest score because it appears to actually show that MMR contributes to higher autism rates.”

The key phrase in the whole site. Data that supports their position is good, data that does not is bad. What makes a study good is not its methodology or its rigor, or its reproducibility, or its biologic plausibility,  but if it supports vaccines casing autism.

Dr. Wakefield, as has been noted over the last week, had his MMR/autism paper withdrawn from Lancet not for bad science, but for dishonest science.  In medicine you can be wrong, but you cannot lie.  If the results of medical papers were shown to be fabrications, such as the papers of Scott S. Reuben, no one the medical field would defend the results.  Dr. Reuben, as you may remember, was found to have fabricated multiple studies on the treatment of pain.  Nowhere can I find web sites defending his faked research.  No suggestions it was due to a conspiracy of big pharma to hide the truth. No assertions that he is still a physician of great renown.   He lied and is consigned to ignominy.   Physicians who used his papers as a basis of practice no longer do so, or so I would hope.

The response to Dr. Reuban is in striking contrast to the defense of Dr Wakefield, where bad research combined with unethical behavior, results in

“It is our most sincere belief that Dr. Wakefield and parents of children with autism around the world are being subjected to a remarkable media campaign engineered by vaccine manufacturers reporting on the retraction of a paper published in The Lancet in 1998 by Dr. Wakefield and his colleagues.

The retraction from The Lancet was a response to a ruling from England’s General Medical Council, a kangaroo court where public health officials in the pocket of vaccine makers served as judge and jury. Dr. Wakefield strenuously denies all the findings of the GMC and plans a vigorous appeal.”

Opinions did not change when the Wakefield paper was demonstrated to be not just wrong but false, the researcher’s behavior unethical, and the study could not be reproduced using similar methodologies.  Instead, the defense of Dr. Wakefield became, well, like a Jim Carrey shtick. The Mask defends retracted autism research. Fire Marshall Bill on the medical literature.  Jenny and Jim’s defense does make more sense read as comic performance art.  Andy Kaufmann would have been proud.

I wonder if the more grounded in fiction an opinion is, the harder it is to change, the more difficult it is to admit error.  I have to admit I cannot wrap my head around the ability of people to deny reality.  It is the old Groucho line come to life, “Who are you going to believe, science or your lying eyes?”

So I will, I hope, keep changing my mind as new information come in.  It is what separates real health care providers from acupuncturists and homeopaths and naturopaths and anti-vaxers.  It is what some truly great minds admit to doing.  As one deeper thinker and better writer than I said, kind of,

“The other terror that scares us from self-trust is our consistency; a reverence for our past act or word, because the eyes of others have no other data for computing our orbit than our past acts, and we are loath to disappoint them.

But why should you keep your head over your shoulder? Why drag about this corpse of your memory, lest you contradict somewhat you have stated in this or that public place? Suppose you should contradict yourself; what then? It seems to be a rule of wisdom never to rely on your memory alone, scarcely even in acts of pure memory, but to bring the past for judgment into the thousand-eyed present, and live ever in a new day. In your metaphysics you have denied personality to the Deity: yet when the devout motions of the soul come, yield to them heart and life, though they should clothe God with shape and color. Leave your theory, as Joseph his coat in the hand of the harlot, and flee.

A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines and anti-vaxers. With consistency a great soul has simply nothing to do. He may as well concern himself with his shadow on the wall. Speak what you think now in hard words, and to-morrow speak what to-morrow thinks in hard words again, though it contradict every thing you said to-day. — ‘Ah, so you shall be sure to be misunderstood.’ — Is it so bad, then, to be misunderstood?”

Posted in: Clinical Trials, Science and Medicine, Vaccines

Leave a Comment (28) ↓

28 thoughts on “Changing Your Mind

  1. Fifi says:

    Dr Crislip – “In medicine, the approach is, one hopes, that data leads to an opinion. You have to be careful not to let opinion guide how you evaluate the data. It is difficult to do, and I tell myself that my ego is not invested my interpretation of the data. I am not wrong, I am giving the best interpretation I can at the time. For years I yammered on about how it made no sense to give a beta-lactam and a quinolone for sepsis until a retrospective study suggested benefit of the combination. Bummer. Now when I talk to the housestaff about sepsis, I have to add a caveat about combination therapy. It is why my motto is, only half jokingly, ”Frequently in error, never in doubt.”

    Dr Crislip, thank you for a superb post that is both humorous and a fantastic explanation of how science-based medicine is practiced.

  2. windriven says:

    One of the mantras of critical thinking is that correlation does not equal causation. This study uses death rates among the elderly to assess the efficacy of a vaccine against influenza. Old people die for lots of reasons. Influenza is most active at a time of year when people are exposed to a variety of challenges including climate challenges. Older people may, for instance, tend to keep the heat turned down in their homes for financial reasons and this may stress their cardiovascular systems. My point is that, at least through my non-physician eyes, this study is interesting but doesn’t necessarily say a lot about the value of the influenza vaccine in preventing influenza among the elderly. Is it too high a wall to scale to study the incidence of influenza infection among the elderly who have been vaccinated versus those who haven’t and corrected for by the relative efficacy of that year’s vaccine as gauged by the prevalence of influenza in the general population?

  3. Jojo says:

    Mark, I’d really like to thank you for this post. I’ve been following SBM for a while now, and have learned a great deal. You have really added to my understanding of SBM by presenting your ongoing internal dialog regarding flu vaccination in the elderly. Not only have you highlight just how complex something like this can be, you have also helped to illustrate how studies work over time in changing and refining medical practice. It’s nice to have the process explained in detail instead of just being told what to do.

  4. Fifi says:

    Dr Crislip, a new study seems to suggest that some of the reason there’s been a rise in pertussis rates has to do with an evolution of the bacteria and switch from a whole cell vaccine to one that has fewer side effects but is less effective. I’d be interested in reading your thoughts on this since the rise is often attributed to lower vaccination rates (which may well be a factor too in some nations). I’m by no means trying to malign vaccination since the information seems to indicate that maybe a return to whole cell vaccination would be warranted and that vaccination is effect against certain forms of pertussis, I’m just interested in understanding the actual cause or causes and hearing your opinion on it since you’re obviously concerned first and foremost with practicing SBM and getting to the truth of a matter.

    http://www.cdc.gov/eid/content/16/2/297.htm

  5. daniel says:

    Thank you so much. I’ve been trying to get some kind of response to the one key point from that much-maligned Atlantic article for months. And that point was “why don’t we notice any increase in morbidity when we screw up vaccine manufacture, either through guessing the wrong strains or manufacturing problems?” At least I get the feeling someone is listening.

    (I’ve suggested answers to my own question, too; the biggest is that we are measuring a very noisy signal so it could be doing significant good that we just haven’t been able to find yet.)

    Note that since the flu vaccine 1) works in labs and 2) is very safe, I think we should absolutely stick with it for now. At worst, it’s merely ineffective.

  6. Calli Arcale says:

    Instead, the defense of Dr. Wakefield became, well, like a Jim Carey shtick. The Mask defends retracted autism research. Fire Marshall Bill on the medical literature. Jenny and Jim’s defense does make more sense read as comic performance art. Andy Kaufmann would have been proud.

    Given that Kaufmann got seriously into woo later in life, one wonders whether he’d be more proud of Jenny/Jim putting one over on the world, or more proud of them for choosing the woo-woo path. Probably both.

  7. Zoe237 says:

    Does that mean you now like the Atlantic article? ;-)

    I hadn’t heard that whole cell pertussis versus acellular could be responsible for the increase in pertussis rates. Wasn’t DTaP introduced almost 30 years ago?

  8. Lenoxus says:

    Wait, what’s with all this changing-your-mind-as-a-result-of-the-rigorous-evidence stuff? You’re supposed to change it as a result of hearing someone tell you you’re not open-minded, and which crank idea you should immediately accept without dissent!

  9. vexorian says:

    Slightly off-topic: I am shocked to find that something like fourteenstudies.org exists. Its ranking guidelines are so absurd that I guess it is not a surprise a phone survey is their second favorite “study” of all time…

  10. Chris says:

    vexorian, that website is so bad that I believe that three of the doctors on this site wrote scathing critiques between April 13th and 24th of 2009.

  11. BillyJoe says:

    “In medicine we tend to be conservative about changing practice unless there is a preponderance of data to suggest a change is reasonable. Except, of course, if our big pharma overlords take us to a good streak house.”

    Shame on you Dr. Crispin, you should be slowly roasted on a spit, cut into little pieces, and distributed to your graduation class as a warning against the evils of Big Pharma.

  12. Fifi says:

    BillyJoe – “Shame on you Dr. Crispin, you should be slowly roasted on a spit, cut into little pieces, and distributed to your graduation class as a warning against the evils of Big Pharma.”

    Isn’t Dr Crislip merely making fun of rote Big sCAM and Scientologist accusations that Big Pharma controls doctors and all medical researchers? That’s certainly how I read that, not as an endorsement of Big Pharma. Dr Crislip, what did you intend to communicate here? Was the study you’re referring to done by Big Pharma or medical researchers who don’t have commercial interests in the results of the study?

    Billy Joe, are you proposing that Big Pharma doesn’t perpetuate pseudoscience and use unethical marketing tactics that go against SBM to try to influence doctors? Are you conflating the practice of ethical medicine with uncritical support of the commercial enterprise that is the pharmaceutical industry? Do you conflate all medical research with the for-profit research done by Big Pharma? Just curious because it seems damaging to both the practice and perception of SBM to do so.

  13. Mark Crislip says:

    Except, of course, if our big pharma overlords take us to a good streak house.

    see

    http://www.sciencebasedmedicine.org/?p=3650

    http://www.sciencebasedmedicine.org/?p=60

  14. Fifi says:

    Thanks Dr Crislip – I particularly appreciated your post “A foolish consistency”. Bias, it’s not just something that happens to other people. I appreciate your clarity on these issues….but that may be my own bias at play ;-)

  15. BillyJoe says:

    Fifi,

    “Isn’t Dr Crislip merely making fun of rote Big sCAM and Scientologist accusations that Big Pharma controls doctors and all medical researchers? ”

    In fact, they have an influence much greater than most doctors are prepared to admit. But altmed practitioners and supporters go too far. They say there is a conspiracy between doctors and Big Pharma to confuse, misinform and defraud the public

    “That’s certainly how I read that, not as an endorsement of Big Pharma.”

    Yes, Mark was being humourous, ironic and subtlely sarcastic all at the same time. At least that’s how I read it.
    My response was meant to be similarly tongue in cheek. ;)

    “Billy Joe, are you proposing that Big Pharma doesn’t perpetuate pseudoscience and use unethical marketing tactics that go against SBM to try to influence doctors? Are you conflating the practice of ethical medicine with uncritical support of the commercial enterprise that is the pharmaceutical industry? Do you conflate all medical research with the for-profit research done by Big Pharma? Just curious because it seems damaging to both the practice and perception of SBM to do so.”

    No, no, no, and no,
    I’m sorry, I thought my humour would have been obvious enough not to have to use a smily. :(

    regards,
    BillyJoe

  16. Mark Crislip says:

    I put “Except, of course, if our big pharma overlords take us to a good streak house.”

    in sarcasm tags but it wasn’t published that way.

    perhaps it should have been irony tags.

  17. BillyJoe says:

    Mark,

    “In medicine we tend to be conservative about changing practice unless there is a preponderance of data to suggest a change is reasonable. Except, of course, if our big pharma overlords take us to a good streak house.”

    Okay, I think I did misread this.

    I thought it was a reference to the accusation of the altmed sympathisers who assert a conspiracy between doctors and Big Pharma.

    Instead it seems you were referring to the other point I made above, that the doctors are in denial about the influence of Big Pharma on their prescribing habits:

    http://www.sciencebasedmedicine.org/?p=60

    For those who haven’t followed the link and want a brief summary, this bit is probably worth repeating:


    - Pharmaceutical companies give misleading and biased information.

    - Physicians prescribing habits are influenced by drug company interactions.

    - Pharmaceutical reps and pharma sponsored lectures are often the number one source of continuing medical education.

    - Small gifts of food, pens, and other paraphernalia create an obligation that alters prescribing behavior.

    - The conclusions in published studies in peer review journals are in part determined by who provided the funding, and the more pharmaceutical funding, the more like the results will be in favor of the pharmaceutical company’s products.

    - Physicians often do not know when they are being manipulated.

    - Physicians deny that these interactions actually alter their practice. To quote one abstract “Although each physician is likely to consider himself or herself immune from being influenced by gift giving, he or she is suspicious that the “next person” is influenced.

    Again, sorry for the misunderstanding. :(

  18. BillyJoe says:

    “I put “Except, of course, if our big pharma overlords take us to a good streak house”… in sarcasm tags but it wasn’t published that way… perhaps it should have been irony tags”

    …..ah, now I don’t know which one you meant. Perhaps you rolled both into one! :D

  19. Fifi says:

    BillyJoe – I got Dr Crislip’s humor because of the overriding tone of his article, I just didn’t get yours and it seemed to me as if you were making a joke but had taken away a different meaning from what Dr Crislip wrote than I had. Sarcasm just doesn’t read very well over the internet most of the time I’m afraid (not that online communication isn’t fraught with all kinds of opportunity for misunderstandings). You seemed to be implying that warning about the evils of Big Pharma would be silly. Thanks for clarifying.

    Dr Crislip – I got your humor on the first read and then questioned it because of BillyJoe’s response. I figured you would probably be warning students about the potential evils of Big Pharma since you’re clearly a critical thinker and able to withstand uncertainty without blowing a gasket.

  20. Fifi says:

    Most really funny jokes make light of something true, maybe Dr Crislip’s joke was a two-fer…some sarcasm to address the Big sCAMmers accusations of all doctors being pharma shills and also an ironic aside to how a steak dinner can actually influence some doctors.

  21. Draal says:

    Thanks Mark for the update. I wasn’t crazy after all: http://www.sciencebasedmedicine.org/?p=2084#comments
    :-)

  22. cbamity says:

    Change your mind? Flip Flopper. You should stubbornly hold on to your beliefs as if your life depended on them… and don’t let facts get in your way. Filter the world through a warped world view so that everything confirms your way of thinking.

  23. antipodean says:

    A Streak (sic) house?

    That’s a bit undergraduate isn’t it, Mark? You’re supposed to be the sober and august physician type, not running around naked.

  24. anoopbal says:

    Hi Mark,

    I had a question: Why is that you need different studies with different methodologies to really cement a conclusion?

    An author when he picks a method should pick the best one and should provide evidence for the validity and reliability of the methodology used in that study right? Or when we say different methodology are we talking about different labs testing it?

    what is the major reason why it should be tested couple of times to conclude the results?

    Thanks

  25. DREads says:

    Dr. Crislip, thanks for the thought provoking post!

    anoopbal, It is reassuring when different, independent groups of scientists arrive at the same conclusion using different methodologies. Labeling a methodology as “best” is probably not meaningful. Scientific questions are often too complicated to be answered by a single “magic bullet” experiment. In order to eliminate confounding variables, often a question must be broken down into smaller questions in order to provide answers that are independently measurable and reproducible. There are often many good ways to design a large, scientific study, which means independent groups of scientists will probably never design a study in exactly the same way (sometimes, there is variation in the formulation of a scientific question, and when careful, such variations can lead to insights and ultimately improve a theory). If two or more studies lead to the same conclusion, this reduces the chance that a conclusion is the result of a design error. It also helps demonstrate reproducibility, a fundamental part of maintaining the integrity of science. Additionally, earlier studies may have limitations. When less is known, it takes time for a body of knowledge to accumulate before the most useful questions are formulated (i.e. questions whose answers are directly useful for diagnosing, monitoring, or treating disease). And once formulated, it can be a significant undertaking to come up with answers to them. Theories are rarely static, they evolve over time as more research is undertaken. Sometimes, we are fortunate to have enough evidence that a theory becomes somewhat “concrete”, i.e. it is hard to change the theory since so much is already known backed up with a strong body of evidence. When this happens, it is often not resourceful to retest the same questions so we can move on. This does not mean an area of research should be abandoned but new studies should hopefully lead us to learn something new and complementary (not in a CAM sense).

    My two cents,

    Damian

  26. Fifi says:

    DREads – Thanks for the wonderful explanation of research methodologies, it makes a nice compliment to Dr Crislip’s post.

  27. anoopbal says:

    Hi Damian,

    Thanks for taking the time!!

    Could you give me an example.

    For example, let’s try a new drug to increase muscle loss and we measure muscle mass by biopsies. And the sample size is big enough, college aged individuals, Caucasian, 4 months duration, double blinded and so on. Hypothetically, if the study shows it increases muscle mass in these individuals compared to a placebo, can we say such and such drug increases muscle mass in Caucasian college individuals?

    Or do we have to wait for a different group to reproduce the results? And if you were doing the study, what would you change in the second study to confirm the above results?

  28. anoopbal says:

    Anyone who would like to answer my above question?

Comments are closed.