Articles

Deadly Indeed

There are sources of information I inclined to accept with minimal questioning.  I do not have time to examine everything in excruciating detail, and like most people, use intellectual short cuts to get through the day.  If it comes from Clinical Infectious Diseases or the NEJM, I am inclined to accept the conclusions without a great deal of analysis, especially for non-infectious disease articles.  Infectious disease publications I have to read more closely; its part of passing as an expert.

Outside of medicine, I am predisposed to accepting at face value many of the articles in Skeptic and Skeptical Inquirer. They are trusted sources.  Some topics, like haunted house or Big Foot investigations, I barely skim. After all these years, I doubt there will be any new insights into the subject.  Other topics, depending on my interest, I may read more carefully.

I  often read longer articles  many times.  First a quick skim to see if it offers anything of interest.  If it does, then I may read it carefully.

This months Skeptical Inquirer had an article called  Seven Deadly Medical Hypotheses by Reynold Spector.  Just seeing the title and knowing the magazine, I was primed to accept the content at face value.  I enjoy a well reasoned, thoughtful rant. I relish a clever diatribe, even if I do not agree with the topic.   So I gave it a quick skim.  I was discomfited.  My first gut check was ick.  But I was uncertain why.  So I read it slowly and carefully, and still ick.  But why?There is a degree of self absorption in being a blogger.  I can write about what I want any way I want (I remain amazed at how much I can get away with).  The process of writing about a topic helps me clarify in my own mind issues with articles.

The author of 7, as I shall refer to the article,  has over 200 published articles, is a former executive vice president in charge of drug development at Merck and oversaw the development of 15 drugs and vaccines.  I am nobody from nowhere who just takes care of infected patients for a living.  He wins the argument from authority; I am the E. coli evaluating the human.  Oh well,  this is more an exercise for me to enlighten myself; you are the innocent bystander.

Overall the tone of 7 ? It reminded me of the Health Ranger at NaturalNews.com. Really.  Lots of dramatic statements, no qualifiers, no buts, no subtlety, no nuance.  To me, what marks good medical writing is an understanding that there is far more grey than black and white and that generally people are doing the best they can within numerous limitations  One of the many characteristics of the Health Ranger is hyperbole without nuance.  The Health Ranger has a belief system and sees the medical industrial complex through that lens; information is used to support a predetermined conclusion.  Health Ranger is a bombastic style that is both self assured and self referential.

Let us see what 7 has to say. It begins

A chronic scandal plagues the medical and nutritional literature: much of what is published is erroneous, pseudoscientific, or worse.

I’ll grant the first.  I am an Ioannidis convert.  The second seems hyperbole and exaggeration. Pseudoscientific?  Like homeopathy, psi and astrology?  Sorry The author is 17 words in and he has lost me.  I already question his veracity and judgement.  I read the literature. Hundreds of papers a month. I know the literature, and Sir, it is not pseudoscientific.  Suboptimal, often, but not pseudoscientific.  The third?  What could be worse than pseudoscientific?  Oh yeah.  Wakefields Lancet article.  But fraud  is a very rare exception in the over 20 million references on Pubmed.  The author’s opening salvo strikes me as someone more interested in polemic than truth. If done with verve and panache, and above all wit, I like a good polemic. Pomposity with hyperbole, not so much, and calling the medical literature erroneous, pseudoscientific, or worse leans towards the latter.

Two major factors account for a large proportion of this problem.  First, many medical and nutritional hypothesis are ill-conceived.

Are they?  Over 20 million references in Pubmed.  A few, perhaps, were ill conceived before they are tested. Say, measles vaccine induced gastroenteritis causing autism?  Not even that.  If approached honestly and competently, it would be a long shot, but you never knows unless you look.   That is what a great deal of medical research is about: looking around to see if an etiology or intervention or medication will be effective.  Most ideas, I would guess, go nowhere.

Second, the methods used are often epistemologically unsound.

Got me there.  What is epistemologically unsound?  Even after looking epistemologically up on the interwebs, I am uncertain what it means.   I expect the comments will school me on the meaning of epistemologically unsound. I guess that is why I am a lowly clinician.

Moreover, the same unsound methods are often repeated multiple times on the same tired hypotheses with the same incorrect results.

Isn’t that three major factors?  Or is that the unsound epistemological I cannot understand?  I shouldn’t quibble about counting, but I feel a rising tide of ridicule and scorn, and I am not one to hold it back.

I am not even done with the first paragraph, and the author has epistemologically lost me. Maybe there is good reason to be unsettled with the article.   And in the first five sentences, there are four references, all to works by the author, to justify the position.  I tend to prefer external references in my literature; the hyperbolic self-validation is what I expect from the Health Ranger and his ilk.  But again, who am I to question (1)?

… there is an epidemic of published studies that do not follow the principles of sound medical science- the principles demanded by the US Food and Drug Administration for the licensure and sale of medications.

Well, most studies are preliminary and exploratory.  The rigor demanded by the FDA is the final step in a long process starting with basic principles and, perhaps, epidemiology.  I can’t imagine we should jump to huge randomized, placebo controlled trials for every therapy and to answer every question.  Seems a wee bit excessive to me. Start small and build.  The downside is that there will be dead ends and false conclusions.  The upside is that in the end, a close approximation of Truth will be determined.

The resulting “findings” of such misleading or erroneous studies are often hyped by the news media on the day they are reported or published without any additional, careful analysis.

Hyped “findings”?  Nothing like that in the first two paragraphs of the this essay. Nope.  Nothing to see there but a well reasoned, careful, nuanced prologue for the body of the  essay.  “I” am always “mistrustful” of people who use “quotes” as a form of “sarcasm” when sarcasm is not used for good “effect” like “humor” because it otherwise comes across as “supercilious”.

Now I am starting to understand my discomfiture. Still, that’s just the first two paragraphs.  The body will better, right?

The author then proceeds to the background of how to do a good study: generate a plausible, testable hypothesis and test it.  He uses the Scandinavian Simvastatin Survival Study as an example of medicine done right (a Merck product if you care) and bemoans that not every study meets this high standard.

Too many published studies fail to adhere to these high scientific standards and lead to faulty, and even dangerous, conclusions.

Which is true and to my mind understandable, since there are not the resources to do perfect studies of every hypothesis.   Not every car is a Lexus, not every restaurant has a Michelin three star rating. You can’t always get what you want (2).  The issue to my mind is not that there are suboptimal studies; they are often used to find search for hypothesis that can be tested in better trials.  A large part of research is flailing about looking for something interesting to investigate in further detail.  Not everyone has the resources to test everything using the ‘hypothetical/deductive method” to answer all out questions, like the FDA demands.  Although this is not always the preferred method of generating ideas to test.  I don’t need quotes to cast aspersions on the validity of information or generate guilt by association.  I have learned a thing or two from reading the Health Ranger.

I wonder how many suboptimal studies it required to get to the point of the Scandinavian Simvastatin Survival Study?  The concepts to be tested did not appear from the void, fully formed.  The author does not, as will be seen, pay attention to the history and context of the evolution of medical ideas.

The author then proceeds to his 7 deadly hypotheses. Well, one deadly,  6 not so much.  But guilt by association is a game played by the author of 7 as well.

1) the investigator does not need a specific hypothesis and/or can use an inadequate method to test hypothesis.

He uses the example of epidemiology generated by case-control and cohort studies (the kind of studies that lead to the simvastatin study) and the effects of hormone replacement therapy.  He points out that these epidemiologic studies, for a variety of reasons, can lead to erroneous conclusions. Fine.  The other option?  With no preliminary studies jump straight to a huge trial?  And sometimes epidemiology can lead to important results: that a certain water pump is the epicenter of cholera or that chimney sweeps have more testicular cancer. Or that lowering cholesterol is associated with a decrease in vascular deaths.

Epidemiology is part of a continuum of understanding and evolution of medical knowledge.  But strawmen are easier to burn than recognizing the stuttering, somewhat chaotic progress of medical knowledge.  If proving a point is more important than understanding complexity, this is how you argue.

He then proceeds to genome-wide association studies (GWAS) that have been a disappointment for  elucidating genetic causes of heart disease and Alzheimer’s. The author considers GWAS a failure.  I suppose if you have a narrow perspective, yes it has been a failure. So far.  Huge amounts of information about the genome have been generated, and I am always a fan of knowledge for knowledge sake.  In the world of infectious diseases, there are single gene polymorphisms in the immune system that can increase or decrease a patients risk for a variety of infections.  Is it of clinical relevance yet? No.  Is it interesting? Oh, yes.  Will it lead to a new treatments and diagnostic interventions in the future? Who knows. But trying new ideas may fail but still  lead to insights that may lead to better interventions. I would wonder what secondary advances in technology and understanding were accomplished as a results of the GWAS studies.

It is like complaining that the Apollo program only put 12 people on the moon so the program is a bust since we are not going to the moon for vacation.  Here is a dirty little secret from a mere clinician.  I learn far more from failure than I ever have from success.  “The most exciting phrase to hear in science, the one that heralds new discoveries, is not ‘Eureka!’ (I’ve found it!), but ‘That’s funny…’ -Isaac Asimov.”  If you are a clinician, it is not ‘That’s funny,’ but ‘Oh shit’ that really drives change and knowledge.

2) If women replace these missing hormones post menopausally with HRT, they will remain “youthful” and not suffer from heart disease, dementia, vaginal dryness, hot flashes, and fractured bones.

I remember the late ’80′s,  a time that was the heyday of HRT, when I was in my internal residency training and discussing the issues at length not only in clinic, but with my mother.  I remember discussing the epidemiologic data and the worries of cancer.  The author states that

…based on these  (biased) studies, false claims were made the HRT protected against cardiovascular disease and dementia.

As if we knew it was false at the time. It was the best guess based on the data, and epidemiology can give insights that can be later confirmed  by better studies.  He also says

“the proponents…ignored the well-documented fact that estrogen is a carcinogen that causes breast cancer that can kill women” and that “HRT caused a 25% increase in breast cancer.”

I do not know where the author was practicing, but I remember talking with patients (I know, flawed memory) and my mother about the relative risks of cancer and fracture from HRT.  And 25%. Increase.  That’s bad.

What was the study?  On “16 608, patients, there were  more invasive breast cancers compared with placebo (385 cases [0.42% per year] vs 293 cases  [0.34% per year]…and the estrogen group had higher mortality (25 deaths [0.03% per year] vs 12 deaths [0.01% per year].”

That is bad.  Equally bad was the way the author presented the data, the same author who complains in the opening paragraphs about complex data being presented as looking “superficially adequate to the unsophisticated reader,” but I know when someone is presenting information in an manipulating manner designed to blow smoke out a usually inaccessible area.

In a section worthy of the Vaccine Council or Dr. Mercola, it sounds like people deliberately ignored cancer risk to push estrogen to kill women.  Someone mention hype?  I know it is important to make a point, but those who were investigating HRT and prescribing it, as I did once upon a time, were doing it carefully and with knowledge that there could be risks.

Information does not exist in a vacuum.  When talking with my patients and Mom in the late 1980′s, I basically said, based on the odds, how do you want to live your life?

Lifetime risk is a useful way to estimate and compare the risk of various conditions. Hip fractures, Colles’ fractures, and coronary heart disease, and breast and endometrial cancers are important conditions in postmenopausal women that might be influenced by the use of hormone replacement therapy. We used population-based data to estimate a woman’s lifetime risk of suffering a hip, Colles’, or vertebral fracture and her risk of dying of coronary heart disease. A 50-year-old white woman has a 16% risk of suffering a hip fracture, a 15% risk of suffering a Colles’ fracture, and a 32% risk of suffering a vertebral fracture during her remaining lifetime. These risks exceed her risk of developing breast or endometrial cancer. She has a 31% risk of dying of coronary heart disease, which is about 10 times greater than her risk of dying of hip fractures or breast cancer. These lifetime risks provide a useful description of the comparative risks of conditions that might be influenced by postmenopausal hormone therapy.

That was the kind of information and conversations about HRT I was having with patients in my clinic as I completed my residency, the years the author was at Merck developing drugs.  Many patients were far more worried about the disability and pain of fractures than they were of breast cancer.

In continued hyperbole that is totally disconnected from what I remember, he calls HRT a “flagrant example of the harm done by straying from the principles of hypothetical/deductive approach and sound clinical science.”

Really?  Did this guy ever take care of patients?  Has he ever had to make decisions based on incomplete information?  We are only into number two of seven and he last lost me with the hysteria.  I wonder how he would suggest exploring the effects of waning estrogen on the health of women?  Jump straight to a large trial?  Do no preliminary work?  Ignore any potential leads?  What is the alternative to the incremental, and sometimes erroneous, results of medical understanding?  How about fluoride and tooth decay?  So many insights start with a guess and a little epidemiology. Sometimes it pans out, sometimes it doesn’t.  But you do not know unless you try.

3)  if small dosages of vitamins are good for humans, very large doses would be better for everyone.

He then notes the studies that show the hypothesis was wrong.  But this was only known after the fact, after the studies,  and perhaps using vitamins like drugs would have beneficial effects.

Then the odd summary: ” megavitamin therapy tested in properly controlled trials either does nothing or is harmful (except in a few well defined exceptions).”

So it does nothing except when it does.  And how would we know the well defined exceptions unless we did the trials?

He goes from complaining about the science to complaining about the regulatory and commercial issues of megavitamins, changing arguments in midstream.  Is it the science or how the science is used?  Two different issues.

This is getting tedious, even for me.  I will soldier on, although the re re re reading 7 is increasingly painful. The closer I read it, the greater the errors and manipulations; a Mandelbrot set of manipulative medical writing.  Soon I will find the indefinite articles and pronouns suspect.  I try to skim the Health Ranger for a few chuckles; that is not why I read SI. And when is their swimsuit issue?  Oh. Wrong SI.

4) Screening tests beyond the standard medical examination are necessary for identifying disease and the risk of disease in apparently healthy, asymptomatic adults.

I will leave this issue to the more knowledgeable hands of Dr. Gorski.  His argument seems to be based on the 20:20 vision of hindsight, which is apparently the primary argument in all seven cases.  We thought screening would be effective,  studies showed it wasn’t, so the hypothesis was flawed and we should not have suggested screening or done the studies.

The author does not show in this, or other examples, why the ideas were wrong in the context of time the ideas were first offered. It is only viewed through the all powerful retrospectoscope that the author finds his deadly hypothesis. It is ever so easy to predict the past.

He also seems to argue that since our understanding of the ramifications of screening are not perfect, they are suspect, referencing himself for issues with PSA and mammograms (1).  The author argues in part that since our understanding is imperfect, it is a deadly hypothesis. I have always been comfortable with making decisions based on incomplete information, as that is the only kind of clinical information we ever have, save for the results of the occasional autopsy.  The perfect always being the enemy of the good.

He also complains about genetic screening. He notes that few people with high risk genes will develop disease and they can’t do anything about it, so why bother?  I wonder if the author has had much direct patient care.  What  most patients dislike is uncertainty about the why of their disease and most prefer as much understanding and certainty about their health as they can gather.  That is why they bother. And todays why bother may be tomorrows critical insight.  I have discussed how the show Connections made an impact on my view of the serendipity underlying advances. It may not be cost effective or useful currently,  the author does note that for some patients (breast cancer) it may have utility. Again, it is a deadly hypothesis except when it isn’t. So much sound and fury.

But how do you know until after you have done all the studies and see what works and what doesn’t?  His argument still seems to be since in some patients genetic testing has been shown to be of no utility, in the past they should not have done the work to show it is not useful. Except where it is.  Sort of like going back in time to kill Hitler as a child because he was found to be evil in the future, even though you could not tell that the babe in the crib was going to be the source of Goodwin’s law. And far worse.

Circular argument much?

I do not get the impression the author is one for thinking outside the box.  Usually new ideas lead nowhere, but again, you never know unless you try. Nothing ventured, nothing gained vrs nothing ventured, nothing lost.  It is often not the results of studies that are the issue, but how they are portrayed in the media, as noted by the author, and, probably not intentionally, his entire article is a superb example of just that concept.  Maybe 7 is really meta.

5) Manipulating one’s nutrition can prevent cancer.

As he says,  “In retrospect,  this hypothesis does not seem plausible.”

The whole crux of almost every one of his arguments. Repeat after me. In retrospect. In retrospect. In retrospect.  In retrospect everything is clear.  I have had MD after my name for 27 years, and I remember the uncertainty and interest in all his 7 mostly not so deadly hypotheses.  In the beginning, it was not so clear as he makes it out to be.  The past is easy to predict.

6)  Personalized medicine will greatly advance medical care.

His argument is the same: it hasn’t worked except where it has.

“Personalized medicine has only been shown to be cost effective in a few well defined situations.”

How did we find these well defined situations?  Doing a ton of studies that show benefit in some cases and none in others.

I think the solution to this problem is being able to see the future and know in advance which research ideas will bear fruit and which will be a bust.  Precognition is apparently the only solution. Miss Cleo may be available to help review research proposals, I understand that her readin’ is free.

7) cancer chemotherapy has been a major medical advance.

Of course, in some cases it has been extremely effective, but the war on cancer has not been what it was promised.  Again it seems his argument is the same hindsight argument:  when cancer therapy has been effective, it is great, and when it is not so good, we should not have done the work to show that wasn’t effective.   Again, I leave the details to Dr. Gorski should he choose to cover the topic.

And of course the author doesn’t have a dog in the fight (and there are those quotes, so commonly used by the dispassionate):

“When one dispassionately weighs the minimal prolongation of ‘good’ life in patients with metastatic cancer versus the very distressing side effects of chemotherapy with ‘targeted’ drugs, the case is close.”

I’m convinced,  He is dispassionate.  And Jenny isn’t anti-vax, just pro-safe vaccine. Here is my hypothesis to be tested.  Anyone who argues they are dispassionate isn’t. They are fooling themselves and trying to fool others with their alleged practice of arei’mnu.  Me? I am never dispassionate; although sometimes I do not care, but there is a difference.

Some of his conclusions are reasonable: we need to do our science as best as we can.

The author argues that all these errors and  expenditures of his 7 mostly no so deadly hypotheses could have “been avoided if the hypothetical/deductive method had been applied rigorously.”  I am not convinced, since most of his arguments are based after the fact.  I would be far more impressed if, by using only the hypothetical/deductive approach (no epidemiology, no early studies, no preliminary clinical data, no basic science) if he would predict 7 hypotheses that warrant jumping straight to large, randomized, placebo controlled clinical trials so beloved by the FDA. The Randi prize awaits.

We all need that god like perfection and prescience, unlike those

“guilty of perpetuating worthless practices include “scientists” who repeatedly employ flawed methods and then publish them, government agencies who fund such practices, editors of journals that publish pseudoscience, the USDA and NCI bodies that perpetuate unscientific regimens…”

My. God.  The Health Ranger was right.  The conspiracy has incorporated itself into every aspect of the Medical-Industrial  complex.  A different conspiracy than the one we get from the woo world, but  everyone is involved.

Putting scientists in quotes. A very Health Ranger thing to do.  I don’t suppose he is referring to the “scientists” at Merck who repeatedly employed flawed methods and then published them.

“Approximately 250 documents were relevant to our review. For the publication of clinical trials, documents were found describing Merck employees working either independently or in collaboration with medical publishing companies to prepare manuscripts and subsequently recruiting external, academically affiliated investigators to be authors. Recruited authors were frequently placed in the first and second positions of the authorship list. For the publication of scientific review papers, documents were found describing Merck marketing employees developing plans for manuscripts, contracting with medical publishing companies to ghostwrite manuscripts, and recruiting external, academically affiliated investigators to be authors. Recruited authors were commonly the sole author on the manuscript and offered honoraria for their participation…

This case-study review of industry documents demonstrates that clinical trial manuscripts related to rofecoxib were authored by sponsor employees but often attributed first authorship to academically affiliated investigators who did not always disclose industry financial support. Review manuscripts were often prepared by unacknowledged authors and subsequently attributed authorship to academically affiliated investigators who often did not disclose industry financial support.”

I see people doing the best they can with the tools at hand.  Mostly honest people (I say mostly not knowing what their IRS forms show), working within many limitations, to advance medical understanding.  They do not deserve quotes applied to their work or the title of pseudoscience.  Not everyone is able to achieve the peerless, perfect knowledge bestowed on  a Professor of Medicine and Merck Vice President.

We need “honest” corporations.  Ironic from a former Merck executive;  casting the first stone and all that. I do not need quotes to show my snotty superiority.  We need better regulation of “unsafe and unproven products.”  Like Merck’s Vioxx?.   Ohhh, snap. The Merck shots are cheap shots,  I know. But they made me laugh, and above all I like to make me laugh. It is all about me.

Like the Health Ranger, I see someone with a bee in their bonnet, selectively and histrionically arguing in circles, hoping that if the same cognitive errors and circular reasoning are repeated they will be believed as fact.  I am not enthusiastic about the conclusions and arguments used, being significantly more flawed than the research he rails against. It is not far in style and content from being in the Natural News.  Science, at least,  is ultimately self correcting.  This article, probably not so much.

Of course, I am nobody from nowhere. Not a professor or scientist or a vice president.  I am a clinician and citizen who has to trust his sources of information.  I was raised to judge a man by the company he keeps.  When the NEJM published garbage on acupuncture, my trust in the Journal fell a notch.  The Lancet has always had a reputation of being flaky, it is part of the British charm and I have never held it against them; I just factor it in when reading a paper.  The Annals of Internal Medicine has been untrustworthy for years. Clinical Infectious Diseases remains unsullied.  Now the Skeptical Enquirer (sic) has slipped a bit as well.  7 was primarily deadly for my confidence in its editors. Oh well, at least I can still trust the material published by DC.

Rationalizations

(1)  Crislip et. al.  I said it here before, so it must be right.

(2) And if you try sometime you find/You get what you need.

Posted in: Epidemiology, Science and Medicine, Science and the Media

Leave a Comment (43) ↓

43 thoughts on “Deadly Indeed

  1. David Gorski says:

    Of course, in some cases it has been extremely effective, but the war on cancer has not been what it was promised. Again it seems his argument is the same hindsight argument: when cancer therapy has been effective, it is great, and when it is not so good, we should not have done the work to show that wasn’t effective. Again, I leave the details to Dr. Gorski should he choose to cover the topic.

    Yeah, I’m sure that massive increase in the survival for childhood cancers, leukemias, and lymphomas, all of which are treated primarily with chemotherapy, is an “utter failure.”

    Actually, I must thank you, Mark. I was thinking of taking on Dr. Spector’s most recent article as well. Even if I want to avoid duplication, I think you might well have given me the excuse I need to do something even more useful: Revisit Dr. Spector’s Skeptical Inquirer article from about a year ago that proclaimed the war on cancer to be a miserable failure. I meant to take it on at the time and now forget why I never did. Other topics came up, and by the time I got back to it the article was old news, probably. I do remember being very irritated at the time, but I’m even more irritated by #7 of the “seven deadly medical hypotheses,” which truly does remind me of material I’ve seen on NaturalNews.com in that it’s basically an anti-chemotherapy rant very much like what I see on alt-med websites. The worst part is Dr. Spector mixes the odd good point about medicine with drawing the wrong conclusions from “well, duh!” observations, and then sprinkles in some observations that are so obviously nihilistic and designed to put the worst possible spin on the evidence that they end up being so wrong they’re not even wrong. And, you’re right. Even though many of the facts used are verifiable, it’s the way they’re put together and the absolutist tone of the overall article that conspire to produce a misleading impression. Most irritating of all, though, is Dr. Spector’s blithe assertion without evidence that if only we had “followed the deductive method,” all of these medical missteps could have been prevented.

    Poppycock.

    Most of all, I’m very disappointed in the editor of Skeptical Inquirer. At least these two articles (the article about the war on cancer and this most recent one) are what I like to call “pseudoskepticism.” (I can use quotation marks, too.)

  2. Interesting that someone who gets so exercised about respecting the principles of scientific research can’t even spell the word.

    The “principals” of scientific research presumably are lead authors.

  3. Daniel M says:

    Dr. Crislip,

    I love the Skeptical Inquirer, but that article seemed really fishy to me as soon as I skimmed it. I’m not a doctor yet (hopefully I’ll be starting medical school this fall), but even I saw a lot of hyperbole and alternative medicine-like sentiments in it. Thanks for setting the record straight and confirming that I need to approach even the Skeptical Inquirer more skeptically. Keep up the good work!

  4. David Gorski says:

    @Alison

    Failing to catch that was an editorial failing, not a failing on the part of Dr. Spector. The article is not available online yet; so the excerpts actually had to be manually typed from the article.

  5. windriven says:

    Epistemologically unsound suggests to me that the studies do not follow basic principles of the scientific method or commonly accepted practices of methodology or statistics.

  6. windriven says:

    @Alison

    “And there was this – there is an epidemic of published studies that do not follow the principles of sound medical science- the principles demanded by the US Food and Drug Administration for the liscensure and sale of medications.”

    Liscensure?

    I agree with Dr. Gorski that these are editorial failings. But one wonders if editors spent any time at all with this article before it was published.

  7. David Gorski says:

    Enough spelling flames, please. They only derail the conversation, and, in my experience, contribute nothing of substance. This is a blog, which means there’s very little time to examine posts before they “go live.” If someone gives us a pot of money, perhaps we can manage to make sure every post is perfect before it goes live. In an imperfect word, where most of us have demanding day jobs, sometimes posts are not “cleaned” up until after the post goes live. This has been explained many times before.

    Now…do you have any substantive comments about the post or about Dr. Spector’s article?

  8. My apologies. If I’d realized it was retyped I wouldn’t have said anything. (We love Mark Crislip, but not for his spelling.) I only brought it up because it was odd that someone who kept coming back to one particular concept didn’t know how to spell it. It would be like Mark Crislip consistently writing about killing bigs. Once is a typo; twice raises questions.

    Anyway. Go ahead and delete my comment.

  9. David Gorski says:

    Nahhh. I rarely delete comments here, although I do occasionally make an exception for comments that contain nothing more than a spelling, grammar, or style flame if I sense that it’s derailing the conversation.

    Now, back to the actual post. Or to Dr. Spector’s article, if you happen to have a subscription to Skeptical Inquirer. (It’s annoying that SI doesn’t put the articles for its current issue online, although I can understand why they would do that.)

  10. This isn’t substantive, per se, but appreciative. I had a “huh?” when I skimmed the SI article the other day, but being even further from expert than you (I’m a musician), I didn’t take the time to pick it apart. Thanks for the analysis.

    Also, thanks for recognizing that his association with Merck doesn’t necessarily have anything to do with his arguments. You saved yourself by acknowledging you were making “cheap shots.” It was weakening the post, but indeed I understand that sometimes taking pot shots is just too darn fun to resist….

  11. David Gorski “Enough spelling flames, please.”

    Does this mean that I can’t make my crack about how we should be happy that they studies aren’t episiotomologically unsound?*

    Oh well, I’m sure I’m being pedestrian, but I looked it up, so I may as well paste it…

    “Epistemology (About this sound pronunciation (help·info); from Greek ἐπιστήμη – epistēmē, “knowledge, science” + λόγος, “logos”) or theory of knowledge is the branch of philosophy concerned with the nature and scope (limitations) of knowledge.[1] It addresses the questions:

    What is knowledge?
    How is knowledge acquired?
    How do we know what we know?”
    wiki-esk

    *It’s unbelievable the amount of planning and research I have to put into a stupid little joke like that. I hate to waste the effort.

  12. Also, I wanted to add how much I enjoyed the article.

    My only small complaint is that Miss Cleo isn’t Mrs Cake, but I realize we can’t have TP 24/7

  13. windriven says:

    @Dr. Gorski

    This is a blog, which means there’s very little time to examine posts before they “go live.”

    It was unclear that the quotes were manually replicated. Like Alison, I supposed that they were copied and pasted from an online iteration.

    That said, misspellings in a magazine – as opposed to a blog – are absolutely material. They are a marker for the degree of editorial scrutiny to which an article is subjected. If you read a study in, say, NEJM that had multiple misspellings would you not automatically wonder if errors in statistical analysis or even fundamental methodology also slipped past the editors?

    On the other hand, if the misspellings did not appear in the original then the point of our comments is indeed pointless.

  14. Mark Crislip says:

    any spelling errors in the quotes are mine, I am a poor typist and a worst speller, and I since there is not an electronic version of the article, there was no cut and paste but my transcription. I wonder if writing for this blog has unmasked a subtle learning disorder as I never see the spelling/grammar errors despite spending significant time on the essays. Like others, I see what I think should be there, not what is there. Drive me nuts.

  15. David Gorski says:

    That said, misspellings in a magazine – as opposed to a blog – are absolutely material. They are a marker for the degree of editorial scrutiny to which an article is subjected. If you read a study in, say, NEJM that had multiple misspellings would you not automatically wonder if errors in statistical analysis or even fundamental methodology also slipped past the editors?

    You don’t have any real-world experience with journals and editing, do you? At least, you don’t sound as though you do. For most medical/scientific journals, proofreaders are not the same people as the editors and peer reviewers. Indeed, in many peer review forms for many journals, there isn’t even a place for reviewers to comment on spelling errors.

    Let’s put it this way: Editors of scientific journals choose content. They send out material for peer review, collect the reviews, and adjudicate disagreements among peer reviewers. After a scientific manuscript is accepted, it is then turned over to a different set of people, who proofread the manuscript, work with the authors to clarify points that might not be clear, typeset it, etc. Seriously. In every journal with which I’ve ever dealt or to which I’ve submitted manuscripts, two groups of people have very little to do with the other. In contrast, on this blog, posts go up, sometimes at the last minute. It’s a blog. It’s not the same thing as a magazine or a journal. We don’t get paid. We have a shoestring staff.

    Now, enough, already. None of this has anything to do with the content of the post. As I have pointed out, grammar/spelling/style flames contribute nothing other than to derail the conversation off-topic. Please, let’s get it back on track. Discuss the post, or Dr. Spector’s article. Time to move on. We’ve wasted a lot of verbiage over this, which reinforces to me why I find such diversions so irritating.

  16. Jan Willem Nienhuys says:

    (It’s annoying that SI doesn’t put the articles for its current issue online, although I can understand why they would do that.)

    Even old articles are often not online. But they keep old issues in print.

  17. David Gorski says:

    Most of the more recent issues (last few years) seem to be online.

  18. Taylor says:

    Dr. Crislip.

    Thank you so much for your analysis. I had just read the article this morning and I was similarly bothered. (Coincidence??? ;-))

    The whole HRT issue Spector writes about did not agree with my recollection from the discussions about the issue in medical school and residency either. And I always become skeptical when I start to hear hyperbole rather than the measured language you describe.

    That said, I still am a fan of the Skeptical Inquirer, but maybe not as much as this blog.

  19. wales says:

    Haven’t read the article but it’s interesting from a couple of perspectives. 1) We don’t often hear ex-pharma types ranting about medical interventions and products. This could be due to punitive consequences for breaching non-disclosure agreements. 2) MC repeats the point that the “7” points are hindsight rants, and hindsight is often 20/20. I expect that the majority of comments from medical professionals on this site will support this criticism of the “7″. However, speaking as a non-medical professional, the hindsight perspective does not diminish the importance of this information for medical consumers.

    I can deal with the fact that science/medicine is messy, time consuming and full of uncertainty, without shooting the messenger (medical professionals). However, the degree of uncertainty is not always (perhaps rarely) communicated to the patient, which doesn’t allow consumers to make fully informed risk analyses and decisions. Therefore, the take home message for a skeptical medical consumer like myself is to eschew most “elective” medical procedures (including disease screening); get second, third or fourth opinions and do an immense amount of reading and critical analysis regarding any recommended interventions; and cross my fingers if confronted with a life-threatening accident or illness that requires intervention. Just finished Gilbert Welch’s “Overdiagnosed”. Bravo GW.

  20. Harriet Hall says:

    Thanks for writing this, Mark. By coincidence (great minds think alike?) I had just yesterday contacted the editor of SI to express my concerns about Spector’s articles. The editor assured me that in future he plans to submit articles on medical subjects to preliminary review by other MD skeptics. That should fix the problem. As a contributing editor for SI, I have a stake in ensuring its quality, and I really appreciate your criticism.

  21. wales”However, the degree of uncertainty is not always (perhaps rarely) communicated to the patient, which doesn’t allow consumers to make fully informed risk analyses and decisions.”

    I can not say how often the degree of certainty is communicated to patients (probably depends on the skills of the individual doctors and the listening skills of the individual patients). One thing that is relevant to this article and that I have found helpful is to search out health care providers that offer good nuanced explanations with appropriate qualifiers and to eschew ones who don’t.

    I once had a medical provider that told me that my son’s speech delays where not due to his unilateral hearing loss. Ultimately, I could not get over the lack qualifying words in that sentence (and numerous other similar sentences) Maybe this is just me, but I don’t want the “certain”* version of things. I want one that gives me the most relevant information.

    *I don’t even know why I’m using quote marks. I think it’s contagious.

  22. Dpeabody says:

    Great article from mark as usual. I had to double check what I was reading when I saw this in the SI, I even wondered if it was a test to see if we were paying attention.

    (As a side note, it’s usually a bad idea to judge an article based on spelling mistakes. I am a terrible speller and will never notice a mistake in typed writing… I almost feel sorry for those of you that can see the mistakes)

  23. weing says:

    I wonder if his antipathy for HRT was not colored by the fact that alendronate for osteoporosis was a Merck drug? Great post, as usual. Does make me wonder if executives of other corporations consider their 20/20 hindsight to be something special?

  24. Danio says:

    Beautiful post, Dr. Crislip. I’m encouraged by Dr. Hall’s news that the SI editor plans to ‘peer review’ future medical articles–but what about the one (two) that’s already out there? Will he consider publishing a response from medical experts skeptical of pseudoskeptics?

    Re: ‘epistemological’–oy. I keep hearing people using that word in the stupid PoMo context of “you uppity scientists who think you know everything–don’t you know you can’t “know” ANYTHING?”. These encounters are as tedious as they are tortuous. I wrote about one such experience here.
    Constant Vigilance!

    ~JenBPhillips

  25. wales says:

    Weing, without 20/20 hindsight we’d all be doomed to repeating the same mistakes over and over. Not that the hindsight prevents repetitive mistakes, but at least it can inform those who try to prevent the repetition.

  26. wales says:

    Michele, good points.

  27. Joe says:

    David Gorski on 25 Feb 2011 at 11:38 am “Most of the more recent issues (last few years) seem to be online.”

    Many articles from “Skeptic” and “Skeptical Inquirer” are available OnLine, free from home, to people who have public library cards. Check your library system. If not, try your local state college/university, you may have access from public computers at the school.

  28. David Gorski says:

    I was unaware of this. I do note that some SI articles from even recent issues are not available online. One wonders how SI decides which articles will become available online and which will not…

  29. Thank you for your breakdown, Dr. Crislip, it was extremely informative.

    As for Dr. Spector’s use of the term ‘epistemologically unsound’, given his lack of demonstration of *how* those methods are epistemologically unsound, he really shouldn’t be using the term.

    Most scientists aren’t well versed in Philosophy of Science, though many will attempt to dabble. “Epistemologically unsound” is a term often found within a Phil of Sci article, but the onus is on the author to demonstrate how and why the term applies within the particular paper (and this obligation is typically fulfilled in a semi-competent paper, or better).

    Typically this term is applied when someone is attempting to prove something that is beyond the capability of the tool that is being used. For example, a conclusion that ‘the soul does not exist’ derived from a series of MRI studies would be epistemologically unsound. Epistemology is the study of knowledge itself: how and what we can know, or even if we can. The boundary of ‘what can be known’ falls within this purview too. If something is defined as part of World A and World A does not intersect with World B, then any study set up in World B to gain information about World A would be epistemologically unsound: the study attempts to gain knowledge from beyond the limit of gainable knowledge, from somewhere that is (essentially) pre-defined as ‘out of reach’.

    Hopefully that clarifies, rather than muddles. Tis late…

  30. Mark Crislip says:

    brian
    Thank you; that was a explanation that made sense.

  31. Investigator says:

    There are a number of mistaken characterizations here with respect to Reynold Spector’s latest article in Skeptical Inquirer. This is unfortunate because what Spector has to say is significant for rigorous clinical trials. His tone may not appeal to you, but the length of articles and the typical audience of Skeptical Inquirer do not always permit depth and complexity at the academic level. Derogatory comments and comparisons to Health Ranger are inappropriate. Let’s look past his writing style and instead focus on substance.

    MC: I read the literature. Hundreds of papers a month. I know the literature, and Sir, it is not pseudoscientific. . . . But fraud is a very rare exception in the over 20 million references on Pubmed. The author’s opening salvo strikes me as someone more interested in polemic than truth.

    Is it possible that what you read as a clinician is not representative of all published literature in medicine? I suspect it is not. Is it possible that you are skimming so many of these studies you may be missing methodological weaknesses? Most clinicians are not at academic medical centers with unlimited and virtual access to thousands of journals; most do not have the time to practice and read beyond their own specialty. Alternative practitioners, unable to publish in more traditional journals, have created their own to give legitimacy to their research and to justify their practices, much of which could be accurately described as pseudoscience. Then there are the reputable journals that give space to the same thing, for example, the August 2009 issue of the Annals of the New York Academy of Science (Volume 1172) on Longevity, Regeneration, and Optimal Health Integrating Eastern and Western Perspectives.

    Do you really think that only the well-publicized cases of fraud are the only ones that exist? The author is making a generalization based on his experience, and I suggest that if you spent 12 hours a day for a few months reading articles in a broad range of journals for methodological rigor, you might reassess how much of what is published is garbage. It isn’t scientific to simply dismiss his conclusion based on your limited experience, in the absence of a representative literature review.

    MC: Are they? Over 20 million references in Pubmed. A few, perhaps, were ill conceived before they are tested. . . . What is epistemologically unsound?

    The fact that something is published proves nothing. Not everything published is peer-reviewed by investigators familiar with research design or statistical analysis. The fact that something is tested proves nothing if the hypotheses are so ill-conceived that all possible variables are not controlled. Epistemologically unsound in this context simply means scientifically or methodologically weak or invalid.

    MC: And in the first five sentences, there are four references, all to works by the author, to justify the position.

    First, by citing his previous articles in Skeptical Inquirer, it makes it easier for the average reader to find the references since most will not have access to professional journals. Second, if you went back and read his article “Science and Pseudoscience in Adult Nutrition Research and Practice” it explains the evidence that supports his generalizations, without using the space to make the same argument again.

    MC: Well, most studies are preliminary and exploratory. . . . I can’t imagine we should jump to huge randomized, placebo controlled trials for every therapy and to answer every question. . . . there are not the resources to do perfect studies of every hypothesis.

    The point is there are some study designs that are more rigorous and reliable than others.

    MC: The issue to my mind is not that there are suboptimal studies; they are often used to find search for hypothesis that can be tested in better trials. A large part of research is flailing about looking for something interesting to investigate in further detail. Not everyone has the resources to test everything using the ‘hypothetical/deductive method” to answer all out questions, like the FDA demands.

    The point is there are not sufficient financial resources to fund suboptimal studies by flailing researchers that offer methodologically weak results, which require the expense of more studies to validate.

    MC: The author does not, as will be seen, pay attention to the history and context of the evolution of medical ideas.

    Yes, the author is not a historian of medicine, but the point he is trying to make is that we can learn something from past mistakes to improve the quality of future investigations.

    MC: He points out that these epidemiologic studies, for a variety of reasons, can lead to erroneous conclusions. Fine. The other option? With no preliminary studies jump straight to a huge trial? And sometimes epidemiology can lead to important results

    The point is not to undermine epidemiology but to be more cautious about interpreting results from these kinds of studies pending replication by more rigorous clinical trials.

    Spector is attempting to raise awareness that we have learned some research methodologies are stronger than others to avoid repeating past mistakes. Because he is not a historian of medicine, his presentism or retrospective analysis is simplistic, probably for the sake of brevity. Nevertheless, he makes a number of valid points, such as the media giving credibility to any published study, without the capacity to evaluate inherent weakness and limitations. I do not know Spector and do not agree with all of his statements, but I recognize his insights about clinical research and his efforts to promote critical thinking about evidence-based medicine to the public.

  32. David Gorski says:

    Alternative practitioners, unable to publish in more traditional journals, have created their own to give legitimacy to their research and to justify their practices, much of which could be accurately described as pseudoscience. Then there are the reputable journals that give space to the same thing, for example, the August 2009 issue of the Annals of the New York Academy of Science (Volume 1172) on Longevity, Regeneration, and Optimal Health Integrating Eastern and Western Perspectives.

    One can’t help but note that nowhere does Spector mention alternative medicine as an example of “pseudoscience” in the medical literature, something I do all the time. I addressed this in my post about Spector’s article:

    http://www.sciencebasedmedicine.org/?p=11185

    Spector’s full of sound and fury, signifying nothing. He had a chance to make a legitimate and important point, but decided instead to go all polemical instead. More’s the pity.

  33. Mark Crislip says:

    You give more credit to the author than I can find in his text.

    Historicity is key to his arguments.

    His basic argument as I read it is that we should not have done the studies THEN based on the information we have NOW, even though the information we have now is due to the studies we did then.

    He completely ignores the how uncertain issues such as HRT and use of vitamins were at the time. He argues that if the hypothetical/deductive approach had been used, all this time and money would not have been wasted, but fails to demonstrate why, in the context of the time, why that would be so. Cortisol, while not a vitamin, is a example of a natural product that can be used with therapeutic effects. Why not a vitamin as well?

    If the whole thrust is we have “learned some research methodologies are stronger than others to avoid repeating past mistakes.” and to be cautious about preliminary studies. Big duh. That is both banal and, perhaps, counter to human nature. And as if people are not aware of that. What we need, perhaps, is perfect people to apply a perfect science.

    He has good points in the article, but they are lost in the screed.

  34. Investigator says:

    DG: nowhere does Spector mention alternative medicine as an example of “pseudoscience”

    The point is there is evidence to substantiate his statement. Polemic and hyperbole are dismissing his comments as nihilism, as if he is attacking the whole of science and medicine in their entirety. Critical realism is not the same as nihilism, and medical research can take it.

    MC: I can write about what I want any way I want (I remain amazed at how much I can get away with). . . . Big duh. That is both banal and, perhaps, counter to human nature. And as if people are not aware of that.

    Yes, you can get away with it because nobody will care to read what you write. Your repeated condescending remarks are unprofessional. Most people who read Skeptical Inquirer are not professional scientists that they are likely unaware of these methodological issues. There is no reason to think it is counter to human nature to learn something from the history of past research, despite the presentism.

    Reflexive defensiveness is unnecessary. Giving colleagues the benefit of a doubt, scholarly detachment, focusing on substance, and professional courtesy promote dialogue, collaboration, and credibility; otherwise, you alienate allies, undermine your efforts, and simply mirror the sites you ridicule.

  35. Investigator,

    That’s nice, dear.

  36. David Gorski says:

    The point is there is evidence to substantiate his statement. Polemic and hyperbole are dismissing his comments as nihilism, as if he is attacking the whole of science and medicine in their entirety. Critical realism is not the same as nihilism, and medical research can take it.

    Except that “critical realism” is not what Dr. Spector was demonstrating.

    Also, the point is that Dr. Spector went after the wrong target, as I described in my own post. He didn’t go after the pseudoscience of CAM. Oh, no. Rather, he lumped studies that might not have been sufficiently rigorous according to his standards but most definitely do not qualify as pseudoscience together with pseudoscience, which leads me to think that either (1) Dr. Spector does not have a firm grasp of what pseudoscience is and as a result uses the term as an all-purpose pejorative for any science he considers insufficiently rigorous or (2) he doesn’t care and as a result uses the term as an all-purpose pejorative for any science he considers insufficiently rigorous.

    Either way, it’s sloppy reasoning, bad argumentation, and dubious science, for which he deserves to be taken to task. The reason I say he lapses from healthy skepticism into nihilism is because at every turn he cherry picks studies and data to draw the worst possible conclusions and then generalizes those worst possible conclusions to whole categories of hypotheses. His “Deadly Hypothesis” about chemotherapy could have made a reasonable point if he had simply said that in general chemotherapy doesn’t work very well for advanced solid malignancies (which is true) and/or that it is often oversold for what it can accomplish (which is all too often true, depending on the oncologist) and/or that a more selective use of chemotherapy would be a good idea. If he had only said these things, I would have had little quibble with him. I might not have agreed with him, but he would not have provoked the violent reaction he did.

    Instead, he decided to pick the shortcomings of chemotherapy and use them to claim that chemotherapy doesn’t work (except when it does) and to deny that it has been a major medical advance in the treatment of cancer, even though unknown millions survived cancer because of chemotherapy when without it they would most likely died of their disease. Even a few percentage points increased survival adds up to lots and lots of people, after all. There’s a difference between a reality check and denialism, and Dr. Spector’s rant with regard to cancer flirts with denialism.

  37. Mark Crislip says:

    “Your repeated condescending remarks are unprofessional.”
    Snort.

  38. Anthro says:

    Touche, Dr’s Gorski and Crislip!

  39. Ohhh, I’m going to make popcorn. Anyone want to place a bet on who will be the first one to exclaim (an erudite version of) ‘I know you are, but what am I?’

  40. Mark Crislip says:

    I know I am, but what are you?

    Does that count?

  41. Heck, I didn’t even get to collect the bets or put the butter on the popcorn. (grumble)

  42. I completely missed the appropriate answer.

    Mark Crislip “I know I am, but what are you?”

    I’m Juvenile. Nice to meet you.

  43. Calli Arcale says:

    There is little said that I can add to with respect to the article, except to likewise bemoan what seems to be mostly sound and fury in a normally rigorous publication like SI.

    As a former English major I can comment, however, on the typos. This is not a grammar flame, but is intended as encouragement. Dr Crislip, don’t worry about a previously undetected learning disorder; the inability to detect one’s own typos appears to be nearly universal in the human species, and is probably the main reason editors were first invented. You are in excellent company with many of the best authors of all time. ;-) Don’t let it get you down.

Comments are closed.