Oct 01 2012
Countering ideologically motivated bad science, pseudoscience, misinformation, and lies is one of the main purposes of this blog. Specifically, we try to combat such misinformation in medicine; elsewhere Steve and I, as well as some of our other “partners in crime” combat other forms of pseudoscience. During the nearly five year existence of this blog, we’ve covered a lot of topics in medicine that tend to be prone to pseudoscience and quackery. Oddly enough, there’s one topic that we haven’t really written much about at all, and that’s genetically modified organisms (GMOs). GMOs, as you know, are proliferating, and it’s quite worth discussing the potential and risks of this new technology, just as it is worthwhile to discuss the potential benefits versus the risks of any new technology that can impact our health, not to mention the health of the planet. Unfortunately, GMOs have become a huge political issue, and, I would argue, they have become just as prone to pseudoscience, misinformation, and bad science as vaccines, with a radical group of anti-GMO activists who are as anti-science as any antivaccinationist or quack.
Leave it to that quackery promoter to rule all quackery promoters, Mike Adams, to give me just the opportunity to show you what I mean. Over the last couple of weeks, Mike has been in a fine lather about GMOs, with multiple posts with titles such as The GMO debate is over; GM crops must be immediately outlawed; Monsanto halted from threatening humanity and The evil of Monsanto and GMOs explained: Bad technology, endless greed and the destruction of humanity. In other words, it’s a series of post with Adams’ typical hyperbole. If you were to believe him, GMOs are the product of a plot by Satan, Monsanto, big pharma, and the government, and he’s not sure which one of these is the most evil of the bunch.
Not to be outdone, that other quackery supporter vying with Mike Adams to be the quackery supporter to rule all quackery supporters, Joe Mercola, also weighed in with a post entitled First-Ever Lifetime Feeding Study Finds Genetically Engineered Corn Causes Massive Tumors, Organ Damage, and Early Death. It also turns out that Mike Adams had pontificated about this very same study a couple of days before Mercola with a title equally ominous, Shock findings in new GMO study: Rats fed lifetime of GM corn grow horrifying tumors, 70% of females die early. Whenever I see the cranks pile on a study like this, my curiosity is piqued. I also noticed that Steve Novella had already discussed the study that had this not-so-dynamic duo in such a frothy lather. Of course, as you know, that a blogger as awesome as our fearless leader had covered a topic never stopped me from pontificating about the very same study before (well, actually, it has, but in this case it wasn’t enough to stop me). Besides, these sorts of studies are right up my alley, given that I’m a cancer researcher, and the study being touted as “smoking gun” evidence that GMOs are pure evil is such a an incompetently designed and performed study that it actually irritated me more than the usual bit of bad science that I discuss on occasion.
There’s a lot in common between anti-GMO activists and antivaccine activists. Perhaps the most prominent similarity is philosophical. Both groups fetishize the naturalistic fallacy, otherwise known as the belief that if it’s “natural” it must be good (or at least better than anything man-made or “artificial”). In the case of antivaccine activists, the immune response caused by vaccines is somehow “unnatural” and therefore harmful and evil, even though the mechanisms by which the immune system responds to vaccines are the same or similar to how it responds to “natural” antigens. That’s the whole idea, to stimulate the immune system to think that you’ve had the disease without actually giving you the disease, thus stimulating long term immunity to the actual disease! In the case of anti-GMO activists, the same idea appears to prevail, namely that, because GMOS are somehow “unnatural,” they must be harmful and evil. That’s not to say that they might not have problems and issues that need to be dealt with, but the apocalyptic language used by many of the anti-GMO activists like Mike Adams and Joe Mercola is so far over-the-top that it is very much like the language of the antivaccine movement. In fact, not surprisingly, antivaccinationists are often anti-GMO as well, and vice-versa, an example of crank magnetism in action. Indeed, Joe Mercola himself is one of the biggest backers of California Proposition 37, which would require the labeling of GMO-based food, having donated $1.1 million so far.
The particular study that has been reverberating through out the anti-GMO community over the last couple of weeks was done by a group in France led by Gilles-Eric Séralini at the University of Caen with a history of opposition to GMOs. Also, as Steve pointed out, Séralini et al did not allow reporters to seek outside comment on their paper before its publication. If there’s a bigger red flag that a study is ideologically motivated crap and that the authors know it’s ideologically motivated crap, I can’t think of one. Even if Séralini et al didn’t know their study was weak and were somehow afraid that the nefarious Monsanto scientists would plant negative sound bites into news stories about the study, I’m sorry, but trying to control initial news reports like this is just not how scientific results should be announced, period. It’s cowardice and an unseemly attempt at spin:
“For the first time ever, a GM organism and a herbicide have been evaluated for their long-term impact on health, and more thoroughly than by governments or the industry,” Séralini told AFP. “The results are alarming.”
Meanwhile, Mike Adams writes:
As a shocking new study has graphically shown, GMOs are the new thalidomide. When rats eat GM maize, they develop horrifying tumors. Seventy percent of females die prematurely, and virtually all of them suffer severe organ damage from consuming GMO. These are the scientific conclusions of the first truly “long-term” study ever conducted on GMO consumption in animals, and the findings are absolutely horrifying. (See pictures of rats with tumors, below.)
What this reveals is that genetic engineering turns FOOD into POISON.
I do so love all caps for emphasis. So very antivaccinationist-like.
Meanwhile, Mercola writes:
The research was considered so “hot” that the work was done under strict secrecy. According to a French article in Le Nouvel Observateur,2 the researchers used encrypted emails, phone conversations were banned, and they even launched a decoy study to prevent sabotage!
One wonders if they mixed up the “decoy” study with the real study, if the quality of the final published study is any indication. Let’s take a look. This study, by Séralini et al, was entitled Long term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize. Here’s the abstract:
The health effects of a Roundup-tolerant genetically modified maize (from 11% in the diet), cultivated with or without Roundup, and Roundup alone (from 0.1 ppb in water), were studied 2 years in rats. In females, all treated groups died 2–3 times more than controls, and more rapidly. This difference was visible in 3 male groups fed GMOs. All results were hormone and sex dependent, and the pathological profiles were comparable. Females developed large mammary tumors almost always more often than and before controls, the pituitary was the second most disabled organ; the sex hormonal balance was modified by GMO and Roundup treatments. In treated males, liver congestions and necrosis were 2.5–5.5 times higher. This pathology was confirmed by optic and transmission electron microscopy. Marked and severe kidney nephropathies were also generally 1.3–2.3 greater. Males presented 4 times more large palpable tumors than controls which occurred up to 600 days earlier. Biochemistry data confirmed very significant kidney chronic deficiencies; for all treatments and both sexes, 76% of the altered parameters were kidney related. These results can be explained by the non linear endocrine-disrupting effects of Roundup, but also by the overexpression of the transgene in the GMO and its metabolic consequences.
Wow. Sounds really disturbing, doesn’t it? Certainly, at first glance it did to me, but something seemed fishy. Although some have pointed out that the rat strain used (albino Sprague-Dawley rats from Harlan Labs) have a high propensity for tumors to develop as it is, initially I didn’t really consider that as big a problem as some do. You want a certain baseline of tumor development, and it’s not entirely unreasonable to pick a strain that develops tumors at a rate that is frequent enough that it’s likely that the strain will be sensitive to carcinogens. On the other hand, if the baseline rate of developing tumors is high enough, there’s not much room to go up further, and it’s harder to detect effects that result in an increased incidence of tumors. The problem with this particular rat strain is that the rate might well reach that point, which is why the control group size is a really big problem.
Indeed, what seemed fishier to me were two things. First, there were only 20 rats in the control group. In actuality, in practice it was less than that, because the authors looked at both males and females; so there were 10 male controls and 10 female controls, which struck me as a rather small number for a study of this type. Then there were nine other groups, with twenty mice in each group, 10 males and ten females each, making for a very complicated experimental design. Indeed, I agree with Marion Nestle, the Paulette Goddard professor in the Department of Nutrition, Food Studies and Public Health at New York University who supports labeling of genetically modified foods on a national scale, when she said, “It’s weirdly complicated and unclear on key issues: what the controls were fed, relative rates of tumors, why no dose relationship, what the mechanism might be. I can’t think of a biological reason why GMO corn should do this.”
“Weirdly complicated” doesn’t even begin to describe it. I found the experimental design unnecessarily complicated to a ridiculous degree, with too few mice in each group, especially the controls. In fact, these were the groups (the number of animals in the group is in parentheses):
- Controls (20)
- 11% GMO (20)
- 22% GMO (20)
- 33% GMO (20)
- 11% GMO + R (20)
- 22% GMO + R (20)
- 33% GMO + R (20)
- R(A) (20)
- R(B) (20)
- R(C) (20)
The percentage means the percentage of GMO corn in the rat chow, specifically the Roundup resistant strain NK603, and “R” means that Roundup had been applied to the corn. R(A) through R(C) are different concentrations of Roundup in the rats’ drinking water. This is way too many groups to have a high likelihood of producing interpretable data, particularly with only 10 females and ten males in each group. In essence, there were 20 experimental groups with ten rats in each group. Most problematic is the small number in the control group. There’s an old study on this line of rats published in 1979 that looked at the spontaneous development of endocrine tumors. After two years, 86% of male and 72% of female rats had developed tumors of the sort described by Séralini et al. Note that the time period of this 1979 study was the same as that of Séralini et al, two years. In other words, the “treated” rats developed as many tumors as expected for this particular strain of rats allowed to live to their natural lifespanand in fact the control groups arguably had an unusually low incidence of tumors.
Elsewhere, biologist Andrew Kniss ran a simulation (for which he provides the code) based on this study and found:
Let’s assume that the Suzuki et al (1979) paper is correct, and 72% of female Sprague-Dawley rats develop tumors after 2 years, even if no treatments are applied. If we randomly choose 10,000 rats with a 72% chance that they will have a tumor after 2 years, we can be pretty certain that approximately 72% of the rats we selected will develop a tumor by the end of 2 years.
In our very large sample of 10,000 simulated rats, we found that 71.4% of them will develop tumors by the end of a 2 year study. That’s pretty close to 72%. But here is where sample size becomes so critically important. If we only select 10 female rats, the chances of finding exactly 72% of them with tumors is much less. In fact, there is a pretty good chance the percentage of 10 rats developing tumors could be MUCH different than the population mean of 72%. This is because there is a greater chance that our small sample of 10 will not be representative of the larger population.
In other words, large numbers matter. In a group of 10 mice, each with a 72% chance of developing tumors after two years, there’s a much higher chance that the number of rats in the control group that develop tumors will be a number other than 7 (72%). Also curious is that the rate of mortality didn’t appear to be related to the dose of GMO corn. The authors attribute this to the GMO corn being so nasty that it was a “threshold” effect, where the observed effect maxed out before the lowest percentage of GMO corn was even hit, which, if true, would imply that a followup study was warranted looking at, for instance, 0% GMO corn to 11% GMO corn. However, more modeling of the study revealed:
But here’s the important part: Simply by chance, if we draw 10 rats from a population in which 72% get tumors after 2 years, we have anywhere from 5 (“t2″) to 10 (“t1″) rats in a treatment group that will develop tumors. Simply due to chance; not due to treatments. If I did not know about this predisposition for developing tumors in Sprague-Dawley rats, and I were comparing these treatment groups, I might be inclined to say that there is indeed a difference between treatment 1 and treatment 2. Only 5 animals developed tumors in treatment 1, and all 10 animals developed tumors treatment 2; that seems pretty convincing. But again, in this case, it was purely due to chance.
It’s even worse than Dr. Kniss demonstrates.
What do I mean? The investigators measured numerous parameters in each group, some of them at multiple different time points. An experiment with this many groups and this many parameters measured this many times is virtually guaranteed to generate multiple “positive” results. How did they control for all these multiple comparisons? I’ve read the study a few times now, and I still can’t figure it out. An experiment with this many groups in which this many parameters are measured is guaranteed to produce “statistically significant” differences in a number of variables by random chance alone. Heck, in Figure 5, I counted 47 different parameters measured, and in some tables thirteen different parameters recorded curiously as percentage changes. Even worse, for the mortality data (arguably the most critical data), no confidence intervals are reported, and there appears to be no discussion of how the mortality data were analyzed, as Michael Grayer points out in an excellent takedown of the statistical analysis (or, more appropriately, lack of statistical analysis) in this paper. I would only add to this a couple of questions. First, why was there no power analysis reported to justify the number of mice per experimental group and the number of experimental groups chosen? What was the statistical power of this design to detect significant differences? This is some very basic stuff here. Second, who the heck was the statistician on this? He or she should be fired for gross incompetence.
And don’t even get me on the lack of blinding of observers to the identities of the experimental groups. That’s just single blinding, which is the absolute minimum that could be acceptable in an animal experiment. Double blinding would have been better. Apparently, the researchers used neither.
There’s another fishy thing about how the results are reported. Steve Novella noted this, too, but it’s more pervasive than he pointed out. In fact, never before in a scientific paper have I seen a line like, “”All data cannot be shown in one report and the most relevant are described here”—that is, until this paper. Steve wondered whether the authors were cherry picking the results they were presenting. I more than wonder. I strongly suspect. After all, in these days of online publication, that’s what supplemental data sections are for, to publish all the supporting data necessary to convince reviewers and readers, but prioritizing the important data to be shown in the paper. What this paper represents, in fact, is a massive failure of peer review.
Then there are the graphs. I was half-tempted to reproduce the graphs here, but in reality I found Figure 1 (which contains them) so confusing. It consists of six graphs, three for males, three for females, each graph consisting of four curves for different percentages of GMO corn in the rat chow. Not only that, each graph had a shaded area stated to represent the mean lifespan and beyond. But not only that, each graph had an inset graph representing “cause of death” for mice who died before the lifespan of the gray area. That’s basically a total of twelve different graphs, in which it’s hideously difficult to directly compare the experimental groups that I would want to compare to each other. It’s almost as though the authors were trying to make it hard to interpret the results of this study. However, considering that, in essence, this was a study of 20 different groups (two controls and eighteen experimental groups), the results are well nigh uninterpretable to the point of meaninglessness. Besides, Emily Willingham went to the trouble (and it was a lot of trouble, I bet) of graphing the data in a much more standard way that makes it easier to interpret. Guess what? The differences mostly disappear. She also speculates whether BPA was a confounding factor, although I’m not particularly convinced by her arguments for that. She is correct, however, in pointing out how crappy the statistical analyses were and deceiving the graphs were. In fact, if you want an idea of why Figure 1 is so deceptive, you can find it in, of all places, this Tweet. When someone can demolish graphs in a Tweet like this, you know they’re bad.
Finally, there is a question of whether the control groups were exposed to GMO corn. Tim Worstall, a blogger at Forbes.com, looked into the issue of whether there is GMO corn in normal rat chow sold for use in feeding laboratory rats. He contacted Harlan, the company that supplied the rats for this study, and asked about GMO products used in rat chow. The company told him that “we do not exclude GM materials from rodent diets.” He also points out that, if the findings of this paper were accurate, because there is a difference in the use of GMO corn in the U.S. and Europe, we’d expect to see a massive change in the incidence of tumors in this mouse strain in the U.S. but less so in Europe. He has a point, but I think he overstates his argument. If the incidence of tumors in these mice is really 72-86% by two years, it could very well be difficult to detect a significant increase in a number that is already so high. On the other hand, his point that the control mice might well have been exposed to GMO corn is valid. Certainly, there is nothing in the paper that demonstrated that the control group’s feed was free of GMO corn.
The bottom line is that this study is about as bad as studies get. The editors of Food and Chemical Toxicology, the journal in which this pitiful excuse for a study was published, ought to hang their heads in shame. As it was so aptly put:
But it could more simply mean the GM maize and the herbicide had no measured effect, and that is why the dose made no difference. “They show that old rats get tumours and die,” says Mark Tester of the University of Adelaide, Australia. “That is all that can be concluded.”
Indeed. That is about all one can say about the study. Certainly we can’t say whether the GMO maize increased the propensity for tumors. It’s also interesting how the authors included so many photos of the rats and their tumors, photos that quacks like Mike Adams and Joe Mercola eagerly post on their websites, but failed to include photos of the control rats. They got tumors too. Were they qualitatively different in histology or organ distribution or different in size than the tumors in the mice eating the GMO maize? No, including all those photos of rats with tumors served no purpose other than to be sensationalistic and provide shocking pictures for anti-GMO activists to use to scare people.
So why should we care? As I said before, I detest ideologically-motivated pseudoscience and bad science. It’s the same reason I come down so hard on antivaccine “researchers” like Andrew Wakefield, Mark and David Geier, and various other “researchers” who pump out bad studies that support the long-discredited hypothesis that vaccines cause autism or that vaccines cause a whole host of problems. This bad science has real implications, both politically and in policy. Already, Séralini’s risibly bad study has motivated the French government to order a probe into the results of the study, which could result in the suspension of this strain of genetically modified corn. Moreover, one can’t help but wonder a little bit about the timing of the release of this study, given that Proposal 37, which would require the labeling of GMO-based food, is a big issue in California right now, and a study like this might just influence the election.
When it comes to GMO, I don’t really have a dog in the hunt, so to speak, but brain dead studies like this one certainly prod me towards the view that much of the “science” behind anti-GMO activism just doesn’t hold water, and the easy acceptance of such nonsensical results as valid by those who should know better but apparently don’t is just plain depressing. There might be valid reasons to be wary of the proliferation of GMO-based foods, such as concern over the control that large multinational corporations like Monsanto might exercise over the food supply, but the studies purporting to find horrific dangers of GMO-based food strike me as having the methodological rigor of a typical Andrew Wakefield or Mark Geier study—or an acupuncture study. Perhaps that’s why I wasn’t too surprised when one of my readers pointed out that one of the authors of the study is also a homeopath and acupuncturist; so maybe the better comparison to make to this paper would be papers by homeopaths trying to show that homeopathy works. Either way, this is bad, bad science, and it’s sad to see how many people who should know better (but apparently do not) lap it up so credulously while applying much greater skepticism to science that doesn’t damn GMOs as pure poison.
Next up, I anticipate that someone, instead of calling me a “pharma shill,” will call me a “Monsanto shill.” It’s coming. You know it is. Just wait. It’s inevitable. On the other hand, maybe I should look on the bright side. Perhaps I can generate a new revenue stream by adding all that filthy food industry lucre to all the filthy pharma lucre that antivaccinationists and quacks think I’m getting.
Heck, maybe Monsanto will want to hire me.*
100 Responses to “Antivaccine versus anti-GMO: Different goals, same methods”