Articles

Antivaccine happenings ten years time ago

pagehappening

This is about antivaccine happenings ten years’ time ago. Unfortunately, it’s also about antivaccine happenings now. The reason, and what links the two, is that antivaccine happenings, particularly myths, never seem to die. They just keep coming back over and over again. One myth that’s been recycled since at least 2005 is the one that claims that there’s been a study that has vindicated Andrew Wakefield. Stories pop up every so often that look for all the world as though they’re new claiming that the results of Andrew Wakefield’s original Lancet case series has been replicated. Sadly for Wakefield (and happily for the rest of the world), it’s just not true. I have a Google Alert for vaccines set up, so I see these stories when they pop up periodically. Sometimes they even make an appearance on Facebook and/or Twitter as antivaccine memes.

This time around, what’s happening is not exactly the same thing. The “Wakefield was right” news stories appear to arise organically every so often. I have no idea why. This particular story is one that’s being desperately pushed by antivaccine mavens—again. It’s one that has been desperately promoted dating back to a decade ago, as you will see. Unfortunately, like Jason or Michael Myers in a classic slasher flick, just when you think it’s dead is the time to be vigilant because it’s not. I’m referring to a myth that I’ve been covering on and off for nearly a decade that the Centers for Disease Control (the CDC) has been “hiding evidence” that mercury in vaccines is a major cause of the “autism epidemic.” It’s not true, of course. Indeed, my very first post for this blog lo these six years ago referred to the hypothesis that mercury in the thimerosal preservative that used to be used in childhood vaccines as a failed hypothesis. And so it is—and remains.

None of that prevents a very vocal bunch of cranks from trying very hard to exhume the rotting corpse of the hypothesis, hook it up to steampunk-appearing equipment straight out of a Frankenstein movie and fire thousands of volts of electricity through it to reanimate it. This time around, it’s Brian Hooker, PhD, whom we’ve only met very briefly before on this blog. However, those of you who follow certain other blogs (such as my not-so-secret other blog) have met Dr. Hooker, a biochemist who fancies himself a vaccine expert. He’s also known for having presented at the yearly AutismOne quackfest, along with Tim Bolen, clearly one of the crankiest cranks in existence, as anyone who’s been following issues of antivaccine beliefs and quackery is likely to know.

I must admit that, until now, I had been assiduously avoiding this particular topic. I must confess that the reason was because I was so incredibly tired of this particular line of “reasoning,” such as it is, having first dealt with it nine years ago. Unfortunately (for me, not for you I hope) no one at SBM had taken on this particular claim before, which means that I feel as though I have to. I realize that this is a bit repetitive for people who follow me both here and at my not-so-secret other blog, but I thought the issue important enough to risk more repetition than usual. Let’s just put it this way. I hope that I can write this and that you can just post it in response whenever you see this particular claim/meme pop up on Facebook or wherever. I can only hope that it will suffice for a reasonable period of time in the future, so that I don’t have to revisit this topic again any time soon.

So let’s go back a bit. This particular claim – that the CDC has been “hiding” or “covering up” evidence that in reality the mercury in thimerosal in vaccines really has been causing autism all along – has been all over, in press releases, on SafeMinds, on the “boy wonder” Jake Crosby’s blog, and, of course, on the antivaccine crank blog Age of Autism. The origin of this claim appears to be a document obtained by Brian Hooker, whose “mistreatment” at the hands of SafeMinds and the antivaccine luminaries at AoA led to his becoming the Frankenstein’s monster they could no longer control. I’m referring, of course, to the Brian Hooker whose epic fails when it comes to vaccine science go beyond epic.

Let’s go to the press release, straight from Hooker himself:

Dr. Hooker, a PhD scientist, worked with two members of Congress to craft the letter to the CDC that recently resulted in his obtaining long-awaited data from the CDC, the significance of which is historic. According to Hooker, the data on over 400,000 infants born between 1991 and 1997, which was analyzed by CDC epidemiologist Thomas Verstraeten, MD, “proves unequivocally that in 2000, CDC officials were informed internally of the very high risk of autism, non-organic sleep disorder and speech disorder associated with Thimerosal exposure.”

When the results of the Verstraeten study were first reported outside the CDC in 2005, there was no evidence that anyone but Dr. Verstraeten within the CDC had known of the very high 7.6-fold elevated relative risk of autism from exposure to Thimerosal during infancy. But now, clear evidence exists. A newly-acquired abstract from 1999 titled, “Increased risk of developmental neurologic impairment after high exposure to Thimerosal containing vaccine in first month of life” required the approval of top CDC officials prior to its presentation at the Epidemic Intelligence Service (EIS) conference. Thimerosal, which is 50% mercury by weight, was used in most childhood vaccines and in the RhoGAM® shot for pregnant women prior to the early 2000s.

My first reaction to this claim was a massive, massive yawn. Why is that? Quite simply, I thought I was having an acid flashback to 2005, as this bogus argument dates back at least that far. For example, see this post. It’s not about the Verstraeten study specifically per se, although it does mention it as an example of how successive iterations of data analysis can result in an effect size seen in preliminary analyses shrinking away to nothing as successive, more rigorous analyses are performed. It’s a common thing in science, particularly clinical science. Physicians know it and expect it. We’re not surprised by it. Indeed, we have an instinctive tendency to be skeptical of large effects reported in early studies because we know they often dissipate in later studies and analyses. There’s even a term that’s been coined for one type of this observation, the “decline effect.”

Hooker, determinedly oblivious to such considerations, along with the antivaccinationists who have been hatching all manner of conspiracy theories about the Verstraeten study for the last decade, do not know it and do not expect this natural and common phenomenon in science. So when they see it, their natural inclination is to assume that it must be a conspiracy.

Hence the yawn.

Indeed, my yawn was so intense that when a reader wrote asking me to take this on or point her to a debunking of this particular bit of antivaccine nonsense, I demurred at first. Basically, I said that the whole matter bored me and that if I were to write about the Verstraeten study again it would be far more out of a sense of duty than out of any real interest. After all, I never thought that, nearly nine years after this particular conspiracy theory appeared, thanks to everybody’s favorite antivaccine crank, Robert F. Kennedy, Jr., the same misinformation would be reappearing, like some crazed Whac-A-Mole™ that needs to be slapped down yet again. Key to RFK Jr.’s conspiracy theory is the claim that somehow, at a meeting in suburban Atlanta at a conference center known as Simpsonwood, the CDC somehow cooked the numbers to “cover up” evidence that, contrary to CDC assurances, thimerosal in vaccine really was strongly associated with autism. Of course, as explained by Skeptico and myself, RFK, Jr.’s account of what happened at Simpsonwood was shockingly dishonest, as anyone can see for himself if he takes the time to read the entire 286 page transcript.

So what’s got Hooker and his antivaccine buddies’ knickers in a knot? To understand this, you need to understand a couple of things. First, the Simpsonwood conference was all about examining evidence from the Vaccine Safety Datalink (VSD), a collaborative effort between the CDC’s Immunization Safety Office and nine managed care organizations (MCOs) established in 1990 to monitor immunization safety and address the gaps in scientific knowledge about rare and serious events following immunization to determine if there really was a reason for concern about thimerosal in vaccines. Although the decision had been made in 1999 to remove thimoerosal from childhood vaccines, the decision hadn’t been fully implemented yet, and the CDC wanted to determine whether there was any cause for concern. It was hardly the action of a group that wanted to “cover up” anything, particularly the bit about publishing the entire transcript. None of this, however, prevented antivaccine activists, particularly the branch known as the “mercury militia” for its affinity for the set of antivaccine beliefs associated with mercury in vaccines as a cause of autism, from dreaming up all manner of conspiracy theories, with the CDC meeting in a “secret” location to cackle over the “autism epidemic,” rubbing their hands together gleefully as they plotted to create a generation of autistic children. (I exaggerate, but really only slightly.) Of course, it didn’t help that Verstraeten ultimately left the CDC and went to work for pharma, which only fueled the hype.

Various antivaccine groups, from SafeMinds on, have been filing Freedom of Information Act (FOIA) requests for well over a decade. Apparently one of Hooker’s numerous frivolous requests finally panned out. Or so he thinks. What he described above in the press release is an abstract presented by four authors, including Verstraeten, at the CDC’s yearly conference in 1999 for the fellows of its Epidemic Intelligence Service. It was a preliminary study, as such abstracts often report, and it reported a relative risk for developing a neurological developmental disorder, comparing the group with the highest exposure to thimerosal at one month compared to no exposure, of 1.8 (95% confidence interval 1.1 to 2.8). In other words, there was a statistically significant difference, but the error bars almost encompassed one. You can get an idea of how preliminary the report is by looking at the 95% confidence intervals for some of the relative risks for some of the conditions reported: autism (RR 7.6, CI=1.8 to 31.5) and sleep disorders (RR 5.0, CI=1.6 to 15.9). These are the preliminary data in which the RR was reported to be 7.6, the original abstract that was said to be “watered down” to a less worrisome abstract to be presented at Simpsonwood. Sounds damning, right?

Wrong.

The Verstraeten VSD study was always intended to be a two phase study, as Emily Willingham and Lindsay Beyerstein reported. They both explain it well, but I think I’ll go to the source, Verstraeten himself, who wrote a letter to the editor of Pediatrics in 2004 about the study:

Did the CDC water down the original results? It did not. This misconception comes from an erroneous perception of this screening study and other epidemiological studies. The perception is that an epidemiological study can have only 1 of 2 outcomes: either an association is found (or confirmed), or an association is refuted. Very often, however, there is a third interpretation: an association can neither be found nor refuted. Let’s call the first 2 outcomes “positive” and “negative” and the third outcome “neutral.” The CDC screening study of thimerosal-containing vaccines was perceived at first as a positive study that found an association between thimerosal and some neurodevelopmental outcomes. This was the perception both independent scientists and antivaccine lobbyists had at the conclusion of the first phase of the study. It was foreseen from the very start that any positive outcome would lead to a second phase. Whereas the original plan was to conduct the second phase as a case-control study, we soon realized this would be too time consuming. The validity of the first-phase results needed urgent validation in view of the large potential public health impact. Did the CDC purposefully select a second phase that would contradict the first phase? Certainly not. The push to urgently perform the second phase at health maintenance organization C came entirely from myself, because I felt that the first-phase results were too prone to potential biases to be the basis for important public health decisions. Health maintenance organization C was the only site known to myself and my coauthors that could rapidly provide sufficient data that would enable a check of the major findings of the first phase in a timely manner.

Because the findings of the first phase were not replicated in the second phase, the perception of the study changed from a positive to a neutral study. Surprisingly, however, the study is being interpreted now as negative by many, including the antivaccine lobbyists. The article does not state that we found evidence against an association, as a negative study would. It does state, on the contrary, that additional study is recommended, which is the conclusion to which a neutral study must come.

In other words, the first part of the study was the screen, to see if there might have been an effect, in which the study was designed to look for associations that, by the design of the study, would need to be, if found, confirmed in the second phase of the study. In this case, it wasn’t. As I said before, this is not an uncommon thing to happen with studies in medicine. Frequently early phases of studies are positive, because they’re designed to look for associations with high sensitivity but with a tendency for a high rate of false positives. Scientists want to see if something might be there. Then they need to test if any associations they find in a first pass-through hold up to scrutiny. In this case the association between vaccines and autism didn’t, and, because of the design of the study, this meant that the study could neither confirm nor entirely refute an association between thimerosal and autism. The conspiracy-mongering RFK, Jr. made it sound as though phase II of the study was tacked on specifically to try to discredit the findings of phase I, but nothing could be further from the truth. Phase II was “baked in,” included in the original design. The only controversial feature was to use an HMO in Massachusetts to crosscheck phase I results, and this was done out of a sense of urgency to get results, not for any nefarious purposes. This happens sometimes; practicality and external exigencies can result in such decisions on the parts of investigators. There was no cover-up, no attempt to whitewash an association.

Of course, no antivaccinationist is ever going to believe that because the source is Verstraeten and because they don’t understand how often studies in medicine start out promisingly positive, only to have associations disappear as more and more confounders are controlled for and more and more rigorous analyses are done.

None of this stops Jake Crosby from attacking Emily Willingham’s analysis in a fashion that really makes me wonder how on earth he graduated from an MPH program, given how lacking in scientific and epidemiologic understanding it is. He engages in the same sort of conspiracy-mongering that Hooker does, only using an e-mail from Verstraeten in which he does what scientists frequently do: Explores different analyses of the data. It even refers to the abstract he had presented.

Of course, since the Verstraeten study, we have multiple other high quality studies that have failed to find a whiff of a hint of an association between thimerosal-containing vaccines and autism. So even if everything Hooker and Crosby says about the Verstraeten study were true, and the CDC did “cover up” a positive study, it wouldn’t matter because it would be an outlier. The fall of the hypothesis that mercury in vaccines causes autism does not depend on the Verstraeten study. It never did. If it were truly positive for an association between thimerosal-containing vaccines and autism, more study would have needed to be done. If it were correctly interpreted as “neutral,” as Verstraeten puts it, more study would have needed to be done. More study was done and it was negative. The antivaccine movement’s obsessive focus on the Verstraeten study is nothing more than a conspiracy theory, and not a very convincing one at that—except, of course, to the antivaccine movement.

In fact, as I’ve known for years and as Emily Willingham summarizes, none of this is even new information! Seriously. It’s not. Look at slide #41 in this PowerPoint presentation by David Kirby, who in 2004 made his name in antivaccine circles by publishing a book, Evidence of Harm, which claimed that mercury in vaccines was the cause of the “autism epidemic. Actually, never mind. I’ll show you the slide:

the76rr

Notice that it says it was an FOIA (that’s Freedom of Information Act) request that provided the Verstraeten data that reported a relative risk for autism of 7.6 in children exposed to thimerosal-containing vaccines. Everything old is new again.

And again.

And again and again and again and again.

This particular meme isn’t new in 2014. It wasn’t new in 2005. It wasn’t new even a decade ago.

Same as it ever was.

Posted in: Vaccines

Leave a Comment (44) ↓

44 thoughts on “Antivaccine happenings ten years time ago

  1. Laurens says:

    Mercury is still quite popular with homeopaths. They use it to treat all kinds of infections. The more the mercury is diluted and shaken, the more potent it becomes (according to homeopaths). That would make a mercury-based C30 homeopathic remedy a ticking timebomb for children, much more dangerous than thimerosal. So let us start blaming homeopathy for causing autism. If it isn’t the mercury, then let’s blame the arsenic they use (to treat diarrhea; a condition infants often suffer from), or the poisenous plants (belladonna), or the rotten duck liver etc.

    It’s so easy to come up with a new conspiracy theory.

  2. Republicus says:

    “PhD Scientist” is becoming one of those things I use to quickly determine whether someone is pushing woo or not. I’ve noticed that mentioning the education level but not the specialty usually correlates with being a crank.

    1. kia says:

      When we live at a time when celebrity PhDs (supposedly in a scientific field) choose to avoid vaccinating their children because they’re into ‘natural’ parenting, then it’s easy to have to question the merits of any given individual PhD. It’s can’t be used as shorthand for ‘person who actually makes/recommends science-based choices’, that’s for sure. In which case, what does it really mean to have a PhD these days? I know non-scientist bloggers who have more regard for the scientific method and plausibility than a lot of science field PhD graduates. But then Dr Oz shows the same goes for medical doctors as well. Qualifications don’t necessarily mean squat.

      1. Chris says:

        Fortunately, the ones who flog their PhDs/MDs as credentials are only a small percentage of the total. It has been addressed here, somewhat:
        http://www.sciencebasedmedicine.org/luc-montagnier-and-the-nobel-disease/

      2. Chris says:

        An even better article about the well educated becoming cranks:
        http://www.sciencebasedmedicine.org/how-do-scientists-become-cranks-and-doctors-quacks/

        I usually have an issue with engineers who decide to expound outside of their realm of expertise. The worst problem is that in engineering you want to work towards a certain outcome, so you do the research and work towards that outcome. This tends to lead to lots of cherry picking, and forcing the data to that outcome.

        Which is the complete opposite of what one is supposed to do in science.

      3. True enough. A degree in a non-associated field does not incur knowledge of that non-associated field.
        But, they rely upon appeal to authority, where authority does not truly exist.

        As for “natural parenting”, I’ve witnessed real natural parenting firsthand while overseas. With its natural attendant high infant mortality rate. Of course, those practicing it overseas did so out of necessity, a lack of modern medical resources.
        It isn’t all that some woo peddlers make it out to be.

        Would that any educational institution that issued a degree to these idiots would revoke that degree.

  3. Sawyer says:

    “This particular meme isn’t new in 2014. It wasn’t new in 2005. It wasn’t new even a decade ago.”

    I appreciate the write-up Dr. Gorski, but this was all that was needed. Forget about their conspiracy theories, forget about their scientific literacy, forget about putting children at risk – these people are struggling to figure out what year it is. My mother is a first grade teacher and I’ve seriously considered asking her how many of her students have a better mastery of reading dates than Jake Crosby.

  4. Frederick says:

    Maybe there’s a big conspiracy : to build a anti-vaxx mouvement, too abuse people and make them believe in alternative treatment ( that do not work), to manipulate the media, because, true or false, a controversy always sale. So those manipulated people go buy books, and natural pill ( not even fill with the product they claim to contained) or sugar pill. A Evil group of people that want only money! Because, as the Wizard of Oz pointed out, Americans spend 35 billions on alternative medicine, every years, So they have all the interest in the world in manipulating people.

    Any Conspiracy can be turn around, I mean all those conspiracy mongers have interest, money, celebrity, political interest or just Ego, in making those “belief” popular. Alex Jones make like 10 millions a years with his stupidity. The worse part is that, the people who follow them should be insulted, because those “guru” really treat them like imbecile. They can use any logical fallacies them want, the are so sure those followers are stupid, that they will still buy them ( and buy the New super duper book full of TRUTH). This show how much the think of their audience.

    I really don’t like those Zombies myth like GMO, water fluoridation , Global warming or electromagnetic field. The worst part is when a political party welcome those people and agree to fight for them, all that base on lies, misinformation and misinterpretation. In Quebec, all those, except Anti-vaxx and global warming, are defended by Quebec solidaire party, a left wing party. And as a very left wing myself, I found that insulting. Even in my political side of the Fence i have to fight,

    Thank You Dr. Gorski for taking time to use your Brain Shotgun to blow the head of that particular Zombies. Like Homer killing zombie Flanders would say : ” He was a zombie?”

  5. Sawyer says:

    If it hasn’t already been suggested, I’d love to see Emily Willingham write a guest post here at some point. She probably has her hands pretty full with the 10,000 different websites she contributes to, but I always enjoy reading her work.

    1. David Gorski says:

      Of course, for one part of this post I stole shamelessly (with attribution of course) from Emily. :-)

      Unfortunately (for me), I think I have a longer perspective on this particular myth. Unfortunately.

  6. William says:

    @ Dr.Gorski
    Is the Yardbirds album from your vinyl collection?

    1. Sawyer says:

      I’m either a super genius or anti-vaxxers are idiots. I spent about 15 minutes critiquing that “study” several years ago and still remember the major flaws with it to this day. Yet someone that supposedly devotes a large chunk of their life studying problems with vaccines think it has “just come to light”?

      And is that stamp collecting comment just a terrible joke, or does they really not know that an epidemiologist is different from a philatelist?

    2. WilliamLawrenceUtridge says:

      Hi Charles. Your links are hilarious. I particularly like the first one. It opens with this paragraph:

      A study from the 1990s has come to light, proving that compared to unvaccinated children, vaccinated children were more likely to suffer from asthma, eczema, ear infections, hyperactivity and many other chronic conditions. Furthermore, the study identified that there was a ten-fold increase in the incidence of tonsillitis in the children who were vaccinated, and a total lack tonsillectomy operations among the children who were unvaccinated.

      You know what I find particularly amusing? The complete lack of discussion about how unvaccinated children tend to die more from pertussis, measles, mumps, rubella, chicken pox, polio, tetanus, HiB, meningitis, diptheria, etc.

      Oh, yeah, unvaccinated children are way healthier, as long as you factor out their total lack of immunity to vaccine-preventable diseases, diseases that are vaccinated against because of their lethality and penchant for causing permanent disabilities.

      Yeah, except for the deaths, sterility and deafness, unvaccinated children are waaaaaaaayyyyy healthier.

      It’s like saying “skydiving is the safest form of exercise once you eliminate all injuries and deaths from the statistics”.

      1. No, it’s saying skydiving is the safest form of exercise and dismiss or ignore all injuries, attributing them to sudden decelerations from a cause that was not skydiving.

        1. WilliamLawrenceUtridge says:

          Ha! Ten points to the sheik.

      2. Elizabeth says:

        I have seen this time and time again posted by anti vax moms….who take their kids to the allergist once a month and ask 100 times in the same month for a “good home remedy” for their kids exzema and other health conditions.

        “My anti-vax kids will outlive all vaxx kids because they are way healthier.”

        Two posts later “Sigh little Tony broke out in a rash. Any good home remedies my homeopathy mommies?”

        A day later “Going to Johnny’s allergist”

        A week later “So glaaaaaaad I don’t throw food parties. My son is sensitive to gluten, red food dye, blue dye, green dye, air, milk and other dairies and some people just don’t get it!”

        She blocked me after I told her one of her “studies” was bunk. :(

    3. Chris says:

      Wow, that first link is so underwhelming. It includes the IAS with Erwin Alber, and a study by a homeopath!

      The second one is a sales pitch for a DVD from an ex-doctor who sell stuff on the internets. Enough said.

    1. mousethatroared says:

      I wonder if we will see a change in SBM editorials approach or the approach of pro SBM commentors based on this new information?

      That’s what SBM is suppose to be about, isn’t it? discarding disproven strategies.

      1. WilliamLawrenceUtridge says:

        Heh, amusing since that’s Dr. Novella’s post today.

        1. mousethatroared says:

          Yes, I admire him for getting right on the topic.

  7. Greg says:

    Can someone please explain some of the terminology used in this post? What does an RR of 7.6 mean? And what is a Confidence Interval? Thanks in advance.

    1. Sawyer says:

      RR is relative risk of a symptom/disease/something happening compared to a control group. If eating nachos increases your risk of a heart attack by 50%, it would have a RR of 1.5. If something has a RR = 0.9, it means it’s 10% less likely to occur than normal.

      http://en.wikipedia.org/wiki/Relative_risk

      I’m not going to try defining CI because some pedantic statistics nerd will flip out if I use the wrong words.

    2. Andrey Pavlov says:

      I’ll let the pedantic statistics folks have at me, because then I learn something as well. But I’ll attempt a simple explanation for Greg, just to understand the concept rather than the actual statistics behind it.

      Basically a confidence interval tells us how accurate our measurement is. We like to believe that when we measure something it is pretty darned accurate. If you want your 2×4 to be 2.25 meters long you take out a tape measure, mark off two meters, and make a cut. The reality is that your 2×4 will be 2.25 meters long +/- (plus or minus) some amount of error. For most things that most people do that amount of error is small enough relative to the accuracy and size of the measurement that it is negligible. You can build a perfectly good shed with 2×4′s measured this way and say it is a 2.25mx2.25mx2.25m cube. But if you take a precision laser measurement you may discover that your 2m board is actually 2.258 meters long. For all practical purposes that just doesn’t matter.

      But when you are taking measurements of things with less precision and/or doing statistical manipulations you must account for that amount of error and describe it. This relates as well to the concept of significant figures (which at least one regular commenter here still fails to understand).

      So let’s say you look at a whole bunch of 2×4′s and you find that the average length is 2.25 meters. If you used a laser measurement you error would be small and so your confidence interval would be correspondingly small. If you used a tape measure that had markings every 1mm your error would be larger but still small. If the markings were every 10cm then your error would be much larger. Because you must, at some point, take a guess as to what the actual length is. If the tape measure has markings every 10cm and the 2×4 falls somewhere between 2.2 and 2.3 meters, you can’t be precise to say that it is exactly 2.25m. You are guessing that it falls roughly right smack between the two marks. But it could actually be 2.24m and you just guessed wrong. If you were to just say “All these boards average precisely 2.25 meters long” you would be giving incomplete information because we need to know how precise you were able to measure that in the first place.

      So when we do a medical study and say that the relative risk is 1.5 with a 95% CI of 1.3-1.7 we are saying that to the best of our ability to measure the effect the RR is 1.5, but we are 95% certain that the true RR lays between 1.3 and 1.7. There is a 5% chance it could be higher or lower and our techniques and abilities would have fooled us into thinking it isn’t.

      That is why it is important to look at the CI’s (95% is sort of the “standard” level of confidence that we report in scientific literature, but it could be any number and for some applications we need a much higher level of confidence). They tell us what the heterogeneity of the data is. A large CI means that the data was all over the place and that we have less confidence that the number we are coming up with is the true number to actually represent the data.

      Also, if the CI crosses 1.0 then we are not too jazzed about the data. It basically means that we are 95% sure that the data could be showing us absolutely nothing. So something like RR 1.6 (95%CI 1.1-2.3) tells us a number of things; that the data is heterogeneous, that the effect could be very small, that the data is skewed (because it isn’t symmetrical around the mean) but that there is likely to be some effect and maybe it is big. Something like RR 1.1 (95%CI 0.9-1.2) tells us that the data is pretty homogeneous (meaning it overall shows the same thing), that the data is not skewed, and that the effect is quite likely to not be real but there is a small chance it could be, but even if it is it is a small effect size. The first example (depending on the context) could be a good indication to do a better study and see what is going on, the second example a good one to say that we should just move on.

      1. brewandferment says:

        it’s why knowing the open ocean position of an aircraft carrier to the nearest yard is a useless number (ie why we use significant digits)–what’s 3 feet matter on a 900 ft long vessel. Even when docked, its position will change based on tide, wind, etc–you could envision the tidal induced positional changes as the confidence interval. Life gets complicated for the deck crew when the tidal range varies greatly, especially if you tie up at one or the other extreme!

      2. Indigo_Fire says:

        So is the confidence interval merely 2 standard deviations from the mean? Or is it an entirely separate calculation?

        1. Sawyer says:

          For a normal distribution, yes. When you start dealing with other types of distribution the math gets really messy. I think a CI still might technically be 2 SD from the mean, but it’s too hard for me to visualize.

          And kudos to Andrey for not slipping up and saying a 2×4 was 2″ by 4″. I really wanted an opportunity to be able to tell him he was wrong about something. :(

          1. Andrey Pavlov says:

            LOL, you do me much kindness Sawyer. And it seems we crossposted, and your comment and mine are in concordance which makes me happy.

            I am certainly wrong about many things, but I try not to post things if I am not sure. Unlike some of our commenters here if I am not pretty certain about something I am about to write I do something truly extraordinary. It is a secret only a select few know and typically requires Level 10 clearance, but I’ll let it slip anyways: I look it up.

            Now don’t go running around telling everyone my secret! The Reptilian Overlords may punish me for giving such power to mere humans.

        2. Andrey Pavlov says:

          @indigo-fire:

          Not exactly the same, no, but in many cases they overlap.

          Basically a CI can be interpreted in a few different ways depending on whether it is used in a frequentist or Bayesian analysis. Also, an SD from the mean must be symmetrical and a CI needn’t necessarily be so. The CI is a reflection of the variance, but it is not a strict interval that gives you the odds of finding the true value of a population parameter.

          So the real answer is – it depends. In a normalized distribution with a standard population then the CI will be the same as a measure of the SD. But that is by chance of the systems looking the same, rather than a fact that the CI is the same thing.

          The wiki page gives a good overview.

          The example I gave was indeed the one that aligned with the idea of 2 SD’s from the mean. Perhaps a better way to think about it would be that it reflects the idea that all subsequent measures on different populations will fall within that CI. So if you are looking at my example, if you had 10 piles of 2×4′s you would say that all of the means would fall within the CI measured based on the statistical modeling of the pile that you actually did measure.

          (NB: I am not an expert in statistical analysis. I am just a little better at it than your average bear. I am pretty sure I am generally right but there may be some nuance that I am missing or misunderstanding. If so, I welcome any corrections)

    3. Andrés says:

      I must be the pedantic statistics guy… [Dr. Pavlov said:]

      This relates as well to the concept of significant figures (which at least one regular commenter here still fails to understand).

      … or perhaps not since Dr. Pavlov’s opinion is that I am the one.

      @Greg: Significant figures doesn’t have any bearing on the confidence interval of RRs. So I recommend you to skip that part. Confidence intervals are rooted in a frequentist approach so I would skip Bayesian credible intervals altogether. I recommend the learning module by Boston University School of Public Health on confidence intervals that includes a good explanation of how they are computed on the RR case.

      @indigo-fire: The 2SD is applied on anything Gaussian (normal) distributed as the average of a set of values. On the RR case the 2SD is applied to the natural log of the RR since it is approximately Gaussian (normal) distributed. It is explained quite well in those learning modules I have linked.

      1. Andrey Pavlov says:

        Yep. You would be that guy.

        And yes, you still don’t understand sigfigs. I found a handy old blog post that sums it up nicely.

        Significant figures doesn’t have any bearing on the confidence interval of RRs. So I recommend you to skip that part.

        Because they are relevant to statistics and propagate through to the output of statistical analysis, including confidence intervals.

        The fact that you think they have no bearing on CI’s may well explain why you simply cannot grasp why the data on VitC you keep spouting off is meaningless. But considering there is a reasonably sizeable literature on how variance affects the need for sigfigs and how sigfigs affect the reportability of results including the confidence intervals themselves you may find it worthwhile to go back and learn the basics again.

        You are inadvertently correct on one point tangential to what you ar actually saying though. In my overly simplistic example (which I prefaced by saying was overly simplistic) what I was actually alluding to is more accurately called error bars. But just like SD’s and CI’s can be the same thing depending on the type of distribution and analysis being done, error bars and CI’s can also be the same thing. It is true though that the CI for an RR is not actually the same as an error bar so in that my analogy failed. So in many cases sigfigs are not particularly important for CI’s specifically in the case of RR’s, they actually still do have bearing and can indeed be important. Depending on what you are measuring they can be very important and even when measuring binary outcomes (like mortality) you simply cannot have an arbitrary number of significant figures and have them be meaningful.

        But the point was to get across a basic idea and general gist for someone with no idea of what these things actually are.

        1. Andrés says:

          @Dr. Pavlov.

          We humans are a lot of times beaten by our own a priori rationalization of events. We are highly stubborn in not bringing up to date our believes (those things that we have previously come to think as true) when faced with new evidence. Even pigeons do better at this: go, read only the first paragraph and ask yourself what is the probability of the prize being behind your first chosen door.

          Moreover:

          If all you have is a hammer, everything looks like a nail.

          And significant figures is just an instrument in order to not fool ourselves that we are managing more information than we do. Are we forced to use it when computing a confidence interval for a relative risk ratio? Certainly not.

          Let’s say that we have measure the number of deaths (no uncertainty in the measurement there) in two populations (let’s say N=800), one under some treatment and one under placebo. Let’s say we have had both 40 fatalities under treatment and 120 under placebo.

          First, can we estimate a confidence interval for the death probability on both groups without any significant figures whatsoever? Yes, we may compute a confidence interval using the binomial distribution just in order not to have to care about any significant figure thing. We can estimate that both real probabilities will be within (28/800, 52/800) and (101/800,140/800) respectively with at least 95% confidence (both of them will be within their respective confidence interval at the same time with probability 91.24%).

          Second, can we estimate a confidence interval for the relative risk ratio (40/120 = 1/3) without any significant figures whatsoever? Yes. We will have to expand a little those intervals to reach over the confidence requirement giving rise to the following 95% confidence interval (I haven’t checked for it to be the narrowest): (28/140,52/100).

          For anyone interested the Tcl program ratio.tcl to compute those confidence intervals is freely available.

          By the way, the example (Page 2) about confidence intervals of your second link is incorrect:

          x±ts
          20.38±(2.015)(0.565)
          20.38±1.14
          20±1

          because the last final confidence interval should have been either (19,22) or 20±2 since 20±1 → (19,21) doesn’t contain the actual value with at least 95% probability (it is narrower than 1.14∙2=2.28).

          Finally. You seem to have a knack for not comprehending what I write. You have even complained when I have tried to make clear what I have said. So now the good news. In case you try to rebut me again with such intelligent comments like either “blahblah” or “bullshit” I will simply ignore them altogether.

          1. Andrey Pavlov says:

            go, read only the first paragraph and ask yourself what is the probability of the prize being behind your first chosen door.

            LOL. Yes, Andres, I am well familiar with the Monty Hall problem and how that affects statistics.

            Are we forced to use it when computing a confidence interval for a relative risk ratio?

            We are never forced to do anything. We can, in fact, do things the wrong way and publish reams of papers on VitC that way.

            Certainly not.

            Did you not notice that I clearly stated that in some cases using sigfigs can be eschewed? Not because it is incorrect to use them but because in specific cases sigfigs aren’t necessary (though you could still use them if you like).

            None of that changes the fact that you are indeed still wrong about sigfigs as it pertains to our original discussion in the VitC data regarding colds. Yet you continue to try and rebut that by using other examples and not even remotely addressing the true issue of trying to measure the difference between lengths of colds in a VitC vs placebo group. It is nonsensical to try and claim a statistically significant difference in length when that difference is on the order of 4-8 hours. You cannot measure the length of a cold to that level of precision. Period. It is a horrible violation of sigfigs to even pretend to do so. And those numbers are an artifact of using averages that provide arbitrary and unjustifiable precision which then gets translated into the statistics to show a “significant” difference.

            And of course your opening volley is, as ever, stunningly ironic:

            We humans are a lot of times beaten by our own a priori rationalization of events. We are highly stubborn in not bringing up to date our believes (those things that we have previously come to think as true) when faced with new evidence

            I’m not the one hopelessly devoted to the idea that VitC simply must have some profound therapeutic effect that is being overlooked.

            It is you who is using incredibly poor logic and argumentation to try and support a conclusion you already have. To the point of conflating completely different physiological states to try and bolster your case and cherry picking case reports written 70 years ago and not reading the report well enough to see they admitted themselves the findings were invalid, just to name a couple. And yes, being devastatingly wrong on the concept of sigfigs when it comes to measuring the difference in the length of upper respiratory tract infections.

            You’ve had not only myself but MadisonMD and others here give you clear, patient, referenced, evidence based explanation of why your claims are not legitimate and not supported by the evidence. About the very salient points of medicine and physiology that you clearly are not taking into account but are vital to the question. About how you also obviously do not understand how clinical trials work nor how to implement and apply them (as MadisonMD pointed out when you continued to assert that the sepsis and VItC trial at Vandy should be repeated, which is wrong). And you top it all off with a sprinkling of JAQing off and a pinch of faux skepticism to boot.

            Sorry Andres, but your ideas on VitC are simply dead in the water. The fact that you are unbendingly convinced otherwise is what made me laugh out loud when I read your opening two sentences just now. Let me be utmostly clear – pick all the nits you want but you are doing science wrong.

          2. Andrés says:

            Dr. Pavlov said:

            LOL. Yes, Andres, I am well familiar with the Monty Hall problem and how that affects statistics.

            LOL“? (I didn’t include “LOLing” within the other intelligent comments, did I?) And once again you have completely missed my point. It wasn’t about if you know the solution now. The learning, humbling experience happens when failing. If they gave you the opportunity to solve it by yourself and you got to the right answer, I am sure you will be able to study some basic statistics in a quite limited time. Certainly I failed.

            Dr. Pavlov said:

            We are never forced to do anything.

            Seriously? You had already said (my bolds):

            Yes, when you are doing active calculations for metrics specific to the measurement at hand you can retain more than significant figures but then at the end you are required to ditch all the decimals that don’t comport to the worst precision you have.

            Certainly I am not required to ditch those digits when I am interested in estimating the mean value of the difference as I have explained once and again. It is beyond me why do you think that significant figures has any bearing in a situation where we can measure minutes without any practical uncertainty and the uncertainty is not due to measuring time itself but when we do realize cold symptoms has ended: that uncertainty is perfectly modeled by the random variable E in my argument and has nothing to do with significant figures.

            Dr. Pavlov said:

            We can, in fact, do things the wrong way and publish reams of papers on VitC that way.

            Just to clarify. Was I wrong supposing you don’t consider garbage the Vanderbilt University group paper?

            Dr. Pavlov said:

            Did you not notice that I clearly stated that in some cases using sigfigs can be eschewed?

            Certainly I did. However it is quite significant that you have selected to explain uncertainty in a ratio confidence estimation with an example of significant figures that is not related with it at all.

            Dr. Pavlov said:

            I’m not the one hopelessly devoted to the idea that VitC simply must have some profound therapeutic effect that is being overlooked.

            Neither I am hopeless nor you have provided any evidence falsifying a positive effect of intravenous/intramuscular vitamin C in gram doses on viral infections. I have already pointed out the supporting evidence that exists.

            Dr. Pavlov said:

            About how you also obviously do not understand how clinical trials work nor how to implement and apply them (as MadisonMD pointed out when you continued to assert that the sepsis and VItC trial at Vandy should be repeated, which is wrong).

            Are you serious? First, MadisonMD precisely said:

            It would be best to eliminate bias using a RCT design (which I would not object to given preliminary evidence).

            Second, after I said:

            If the proper double blind randomized clinical trial of intravenous vitamin C on seriously ill ICU patients were brought to existence the argument would be more compelling or completely refuted. Granted.

            it was you who answered with:

            Granted indeed, but vastly more complicated than I think you realize. I could write an entire paper on just the confounders, difficulties, and practical limitations of the study you are proposing.

            In computer science this situation would be called a deadlock: neither RCT without more evidence nor any more intervention with retrospective cohort for the control group to supply it.

            Dr. Pavlov said:

            And you top it all off with a sprinkling of JAQing off and a pinch of faux skepticism to boot.

            You are enamoured with the JAQing off thing, aren’t you? I haven’t made a single question in my comment that I haven’t answered myself.

            1. Andrey Pavlov says:

              Andres, you do have some solid understanding and clearly a decent grasp on statistics. You are still missing very important points repeatedly addressed. In many cases you simply completely ignore them (as you just did again). You inappropriately shift the burden of proof to demanding a demonstration of “any evidence falsifying a positive effect of intravenous/intramuscular vitamin C in gram doses on viral infections.” This is fundamentally an incorrect approach and belies how you are indeed hopelessly in thrall of your VitC machinations. You continue to confabulate studies to try and build an evidence base for prior plausibility without realizing that the studies you cite are of almost zero evidentiary quality, addressing completely different topics, or both. This is yet another thing you ignore.

              The waters are muddy, you obfuscate the discussion by focusing on trivialities whilst ignoring the salient points MadisonMD and myself have raised, and overall this generates much more heat than light.

              It should be telling to you that a number of people with relevant expertise do not find your arguments convincing (often for the same reasons, sometimes for differing reasons, but always unconvincing none the less) and that more than just I have considered at least some of your comments to be JAQing off, intentional or not (even though it is clear you still don’t quite understand what that means).

              So with that, I bid you adieu (or adios, if your prefer). Keep on keeping on with your VitC infatuation. Pauling went to his grave doing so, unfortunately not to his benefit.

  8. Michelangelo Bucci says:

    After an exhausting discussion with an antivaxxer I have stumbled upon this paper: http://pediatrics.aappublications.org/content/early/2014/02/25/peds.2013-2365

    It’s really a hopeless battle.

    1. WilliamLawrenceUtridge says:

      Didn’t Dr. Novella do a post on that paper? I couldda swore I read something about it here…

  9. Erwin Alber says:

    Vaccines have never prevented anything apart from health, sanity and common sense.

    http://www.vaccinationinformationnetwork.com/dr-russell-blaylock-vaccine-program-based-upon-nonsense-fear-fairy-tales/

    Here is the real story behind the increase in autism:

    Deadly Immunity
    http://www.vaccinationinformationnetwork.com/deadly-deception/

    The question is not whether vaccines cause autism, but why the people responsible for the autism epidemic and its cover-up are not in jail.

    1. Chris says:

      Here is Mr. Alber, the “World’s Worst Person, trying to tell us vaccines never prevented anything. Let’s see if he will ever return to answer my question about measles.

      I have American Census data on the rate of measles incidence for most of the twentieth century. There is one really big anomaly: the rate of measles incidence in the USA dropped by 90% in the USA between 1960 and 1970. Why?

      Mr. Alber’s task is to explain what that happened without mentioning mortality (deaths), any other country (England and Wales are not part of the United States of America), nor any other decade.

      So do tell us why, with supporting scientific documentation why the rate of measles in 1970 (an epidemic year) was so much higher than 1960. Here is the data:
      http://www.census.gov/prod/99pubs/99statab/sec31.pdf
      Year…. Rate per 100000 of measles
      1912 . . . 310.0
      1920 . . . 480.5
      1925 . . . 194.3
      1930 . . . 340.8
      1935 . . . 584.6
      1940 . . . 220.7
      1945 . . . 110.2
      1950 . . . 210.1
      1955 . . . 337.9
      1960 . . . 245.4
      1965 . . . 135.1
      1970 . . . . 23.2
      1975 . . . . 11.3
      1980 . . . . . 5.9
      1985 . . . . . 1.2
      1990 . . . . .11.2
      1991 . . . . . .3.8
      1992 . . . . . .0.9
      1993 . . . . . .0.1
      1994 . . . . . .0.4
      1995 . . . . . .0.1
      1996 . . . . . .0.2
      1997 . . . . . . 0.1

      Knowing Mr. Alber, we will not hear from him again on this thread. Perhaps much later on another article near the sixty days of commenting is allowed ends.

      1. Chris says:

        Yep. he is just that kind of jerk. Trying to post before commenting is cut off.

    2. Windriven says:

      Smallpox. Polio. Pertussis. If that was all it would be more than enough.

      Douche.

Comments are closed.