Articles

Direct-to-Consumer Genetic Testing: Road Map or Tarot Cards?

A topic of growing interest (and concern) at SBM is laboratory and diagnostic test pseudoscience. Bogus tests are everywhere, and Kimball Atwood recently discussed several of them. But over the past several years, diagnostic tests have emerged that appear to be science-based and offer gene-level insights into your health. And these tests don’t even require a physician’s visit – just a swab of saliva and a credit card get you reams of information on your genetics, traits, and risks of dozens of diseases. It looks like the ultimate in consumer health information, with the potential to offer truly personalized treatment strategies. Companies like 23andme, deCODEme, and Navigenics all promise “genetic insights” to improve your health. How could this information be anything but helpful?

Personalized medicine describes medical practices that use information about a person’s genes, proteins, and environment to prevent, diagnose and treat disease. Science-based practice has routinely incorporated environmental advice (e.g., diet and exercise) into medical management. And there are a number of genetic tests in routine use that are well established, clinical validated, and are highly predictive of future outcomes, such as tests for Huntington’s disease and hereditary breast cancer.

Predicting how well drugs will work is another area of personalized medicine. Cancer cells have specific genetic mutations that lead to uncontrolled growth. Identifying the mutation, and then developing drugs to target it, means more targeted treatments. Trastusumab (Herceptin) works only in patients overexpressing the Her-2/neu oncogene. Panitumumab (Vectibix) and cetuximab (Erbitux) effectively treat colorectal cancer, but only in patients whose tumors have wild-type KRAS oncogenes. And gefitinib (Iressa) a drug that initially didn’t look that promising, seems to offer a much better risk/benefit perspective in patients with EGFR mutations. Tumor testing for these specific genetic markers helps predict patient response, guides drug selection, and spares the costs and consequences of ineffective therapies.

Whole-genome testing, offered directly to the public, is the latest development in personalized medicine. Companies typically offer to test for genetic traits that predict the risk of disease or estimate the effectiveness of drug therapies. In a typical test, up to a million single nucleotide polymophisms (SNPs) may be analyzed. A customized report is generated, describing the SNPs and your individual risk factors, usually based on population-level studies of what these SNPs mean.

But our understanding of the correlation between SNPs and the risks they identify is still in its infancy. It’s fair to say we only have a preliminary understanding of the relationship between SNPs, environmental factors, and other influencers of illness. It’s not even clear that SNP variants provide meaningful information on risk, beyond what we already know. For example, a study published earlier this year concluded that adding ten additional genetic variants to to a standard breast cancer risk assessment model had little effect on improving the accuracy of predictions. It adds to a growing list of variants which don’t seem give us any actionable information. But the science doesn’t seem to be stopping the vendors of these tests from making unsubstantiated claims about their accuracy and relevance.

The GAO and the FDA

The US Government Accountability Office (GAO) recently released the results of an investigation into 15 companies offering DTC genetic testing. And it raised some serious questions about the accuracy and validity of information currently available to consumers. The GAO purchased ten tests from four companies, and sent duplicate samples to each company, changing the demographic information each time. The results were alarming:

GAO’s fictitious consumers received test results that are misleading and of little or no practical use. For example, GAO’s donors often received disease risk predictions that varied across the four companies, indicating that identical DNA samples yield contradictory results. One donor was told that he was at below-average, average, and above-average risk for prostate cancer and hypertension.

and

Although the experts GAO spoke with believe that these tests show promise for the future, they agreed that consumers should not rely on any of the results at this time. As one expert said, “the fact that different companies, using the same samples, predict different directions of risk is telling and is important. It shows that we are nowhere near really being able to interpret [such tests].” GAO also found 10 egregious examples of deceptive marketing, including claims made by four companies that a consumer’s DNA could be used to create personalized supplement to cure diseases. Two of these companies further stated that their supplements could “repair damaged DNA” or cure disease, even though experts confirmed there is no scientific basis for such claims. One company representative even fraudulently used endorsements from high-profile athletes to convince GAO’s fictitious consumer to purchase such supplements. Two other companies asserted that they could predict in which sports children would excel based on DNA analysis, claims that an expert characterized as “complete garbage.” Further, two companies told GAO’s fictitious consumer that she could secretly test her fiance’s DNA to “surprise” him with test results–though this practice is restricted in 33 states. Perhaps most disturbing, one company told a donor that an above average risk prediction for breast cancer meant she was “in the high risk of pretty much getting” the disease, a statement that experts found to be “horrifying” because it implies the test is diagnostic.

To be clear, this was not a scientific study, and the results can’t be interpreted to be a comprehensive analysis of the state of the marketplace. But they do give consumers and health professionals cause for concern. Not only did the investigation raise questions about whether these tests are are accurate, there is mixed information about what the results actually mean. The marketing goes well beyond the robust evidence that’s needed to draw stronger correlations between SNPs and specific risk factors. And while some companies are offering legitimate tests (albeit of questionable use), other companies are selling snake oil diagnostics, wrapped in a double helix.

Not surprisingly, the FDA recently took action, sending warning letters to several companies, informing them that these tests require regulatory approval before they can be marketed. While the letters and the FDA’s intent has been subject to considerable scrutiny, the message is clear: the FDA intends to regulate this marketplace.

Expert Opinion – Also Pessimistic

If anyone should be enthusiastic about genetic testing, it would be Craig Venter, who was the first to sequence the human genome. Yet in an intriguing interview in Der Speigel magazine, he is decidedly skeptical about its utility, as noted in a few exchanges:

SPIEGEL: Why is it taking so long for the results of genome research to be applied in medicine?

Venter: Because we have, in truth, learned nothing from the genome other than probabilities. How does a 1 or 3 percent increased risk for something translate into the clinic? It is useless information.

SPIEGEL: Do you think there will be a time when you can extract all this information to yield real medical results?

Venter: For that to happen we need a lot more information: Information about your body’s chemistry, your physiology, your complete medical history, your brain and your entire life. We would need to do that a million times on different people and correlate that data with their genetic information.

SPIEGEL: Will that lead in the end to the kind of personalized medicine that genetic researchers have always touted? Each person would get his or her own personal treatment that is tailored precisely to that person’s genetic make-up?

Venter: That was another one of these silly naïve notions that was out there. It’s not, ‘Oh, we know your genome, we’re going to make this drug for you.’ That will never happen. It is more important that you use the information in the genome about your personal risks and reduce them through intelligent behavior.

It’s anecdote, yes. But when one of the world’s leading scientists is this pessimistic, it doesn’t send a positive signal.

The British Approach

In the United Kingdom, the Human Genetics Commission is taking a different approach than the FDA. Rather than move forward with regulatory intervention, it recently published the Common Framework of Principles for direct-to-consumer genetic testing services, which supports industry self-regulation, while focusing on consumer interests. Here are the highlights:

  1. Scope: The framework is intended to ensure good practices; meeting consumer expectations while safeguarding their interests. The principles apply to tests ordered directly by consumers or by a non-medical intermediary.
  2. Marketing and Advertising: Validity claims should be supported by peer reviewed evidence, and references should be provided. This evidence must be presented fairly and transparently (i.e., no cherry-picking).
  3. Regulatory: Evidence for the association between a genetic marker and a disease, condition or trait identified in the test must be provided. “Associations should be validated at genome wide significance level in more than one large case control study and in a cohort of the ethnic/geographic background relevant to the client. The associations should be published in peer-reviewed scientific journals, they should be undertaken in line with the recommendations made in the STREGA statement, and the provider should supply standard references for these publications.” Accepted and transparent statistical methodologies should be documented to document how the risk of the disease, condition or trait was calculated.
  4. Information for Consumers: Clear and transparent information should be provided to consumers about the tests offered, their scientific basis, counselling offered, how samples will be handled post-test, possible consequences to future decisions (e.g., life insurance), etc. Data shall be presented in easy-to-understand formats. If a test provider intends to recommend products such as supplements, then information about lifestyle modification and other strategies should be provided. Depending on the type of test being offered, a health professional should oversee the information provided under this principle.
  5. Counselling and Support: “Where the test is a genetic test in the context of inherited or heritable disorders, that test should only be provided to consumers who are given a suitable opportunity to receive pre- and post-test counselling.”
  6. Consent: “A genetic test should be carried out only after the person concerned has given free and informed consent. Informed consent can only be provided when a consumer has received sufficient relevant information about the genetic test to enable them to understand the risks, benefits, limitations and implications (including the implications for purchasing insurance) of the genetic test.” “Genetic tests in respect of children when, according to applicable law, that child does not have capacity to consent should normally be deferred until the attainment of such capacity, unless other factors indicate that testing during childhood is clinically indicated.”
  7. Data Protection: All genetic information must be protected as per any other medical or personal health information.
  8. Sample Handling: All handling of genetic samples must be carried out in accordance with legal, medical and professional standards.
  9. Laboratory Processes: Laboratories performing tests should be accredited for quality assurance in molecular genetic testing.
  10. Interpretation of Test Results: Depending on the test, interpretation should be carried out under guidance of an appropriately trained and accredited professional. When information is provided related to risk assessment, the differences between relative and absolute risks should be made clear.
  11. Provision of Results: Information should be provided in an easy-to-understand format. When testing for conditions or traits, non-genetic factors should be identified.
  12. Continuing Support: Depending on the test, ongoing access to consultation should be offered.
  13. Complaints: A robust process for dealing with complaints should be in place.

In order for the DTC market to avoid strict regulation, the Framework looks like a good place for the industry to start. It allows full access by consumers to their own genetic information, while requiring a full disclosure when any interpretation of that information is offered.

Towards a Science-Based Approach

Our access to genetic information currently exceeds our understanding of what that information actually means. The science underlying these tests is still evolving, and that means the interpretations of this data will continue to evolve. Until better and more conclusive data emerges to correlate SNPs with actual health outcomes, consumers will be dealing with probabilities that are largely irrelevant for health decisions. Luckily, there’s a more powerful and far simpler approach: You can make far more substantial reductions in your risk of cardiovascular disease, cancer, or diabetes without any genetic testing at all: simply follow science-based prevention and treatment guidelines.

Posted in: Diagnostic tests & procedures

Leave a Comment (19) ↓

19 thoughts on “Direct-to-Consumer Genetic Testing: Road Map or Tarot Cards?

  1. “One donor was told that he was at below-average, average, and above-average risk for prostate cancer and hypertension.”

    *Well that’s because of quantum or chaos or some such. You know, depending upon whether the donor was being observed on the day he sent the test in or a butterfly was flapping it’s wings in Australia, etc. It all make perfect sense if you really understand the science.

    Oh, but the genetics folks* don’t need quantum or chaos, they have their own completely incomprehensible science that can be twisted to sell people things.

    You SBM people are blinded by your “evidence” and “informed consent” and “cost effectiveness”.

    *irony, sorry, insomnia, kinda grumpy and I have a pre-existing dislike for home genetics testing.

    **I don’t mean to criticize all genetic folks. Many are doing excellent work. I’m only saying that it’s a good medium for those who are willing to twist&sell.

    Interesting article. I’m a little unclear on the difference between the US and UK approach. Is it that the UK has a clearer outline of their expectations. Does the UK expect self-enforcement or is there a government agency that will enforce?

    I’d be interested in hearing what SBM authors think of this blog – http://scienceblogs.com/geneticfuture/

    A while ago I was looking for information of genetic testing for my son. Came across the above blog.

  2. Ian says:

    Just let me know when full genetic sequencing is available for a few hundred dollars.

  3. stbloomfield says:

    One thing that has annoyed me about this topic is how it seems every company is lumped into the same category of “snake oil salesmen” (to use a politician’s words). I think 23andme was one of the company names actually identified, but they weren’t one of the bad companies. Which companies were bad?

    And why does the GAO not do a scientific study to determine if something is scientific? The starting point for this discussion was a study that was flawed, which allowed some politicians to grand-stand and make accusations that weren’t based on solid data. If legislation is based on the results of that GAO report alone, then the industry is probably doomed.

  4. Harriet Hall says:

    Thanks, Scott! My SkepDoc column in the next issue of Skeptic magazine will cover this subject, and it is encouraging to have my findings confirmed by another SBM blogger. I found some intriguing examples of the imperfections of direct-to-consumer genetic testing. A blue-eyed man’s test said his eyes were brown. An African American man who wanted to find out what part of Africa his ancestors came from was told he was 57% Indo-European, 39% Native American, 4% East Asian and 0% African! The title of my column is “Genomic Testing: Are We There Yet?” We’re not.

  5. “An African American man who wanted to find out what part of Africa his ancestors came from was told he was 57% Indo-European, 39% Native American, 4% East Asian and 0% African!”

    I heard of this happening to a woman a few years back. She asked some questions and it turned out that she was not genetically african-american at all; her family were a mix that sounds rather like the mix you describe above, had moved into an african-american neighbourhood and the children identified as african-american. They were very surprised to discover that their identity was purely social.

  6. windriven says:

    It is rather troubling to me that genetic testing is routinely used in the criminal justice system to convict, among other despicable types, rapists and to free others. There have, of course, been several high profile cases where crime labs including the FBI’s have been found wanting in the quality of their work.

    Is the problem an inherent unreliability in sequencing technology, failure to properly interpret the sequences, or uncertainty about the link between sequences and disease? Something else?

  7. windriven says:

    @ Alison Cummins

    I wonder if the family you described found their lives changed by the results? I would like to think that racial identity isn’t as pivotal as it was in years gone by. Still, one presumes that there would be some impact in the way they looked at themselves and the world around them. Do you have any recollections about this?

  8. Maz says:

    windriven,

    Genetic tests can look at a whole range of factors. The tests that this post is concerned with seem to be directed towards consumers wishing to determine disease risks and find more effective treatments. There are VERY few single-gene mutations that we can reliably correlate with a disease or the effectiveness of a certain medication. The problem isn’t our ability to sequence or read DNA (though these technologies DO have an inherent error rate), it’s that we don’t yet have the knowledge to interpret the stuff that we sequence.

    DNA testing to prove identity or paternity, on the other hand, is a much more advanced field. Like I said, our ability to analyze someone’s DNA is well established. It’s fairly straightforward to compare genetic material from a crime scene to that of a suspect. Yes, there have been snafus in DNA testing labs. At the same time, you must consider the alternative. Before DNA testing was routinely used in criminal investigations, eye witness testimony was considered the gold standard for placing someone at the scene of a crime. As we’ve learned, eyewitness accounts are probably the LEAST reliable form of evidence possible!

  9. windriven says:

    @Maz

    Thank you for the clarification. I was aware that many diseases with genetic predictors rarely can be attributed to a single gene. And I am certainly aware of the unreliability of eyewitness testimony. But I wonder what the level of uncertainty in ‘matches’ in criminal cases are and how that is communicated to a jury. One cringes at the thought of a guilty rapist being freed or an innocent spending years in prison on faulty evidence.

  10. windriven,

    Sorry, I don’t know how the story developed. There’s the whole family-secrets thing and then the identity question. Identity is often fairly independent of genetics to begin with, but is it possible to divorce them completely? (The answer is clearly Yes as long as you don’t know the genetics.)

  11. Oh, and the story was from 2003. I looked up my reference but it’s a dead link.

  12. # windrivenon 02 Sep 2010 at 4:20 pm

    “It is rather troubling to me that genetic testing is routinely used in the criminal justice system to convict, among other despicable types, rapists and to free others. There have, of course, been several high profile cases where crime labs including the FBI’s have been found wanting in the quality of their work. ”

    Windriven – I thought you might be interested in this website
    http://www.innocenceproject.org. Here’s a quote regarding DNA

    “Unvalidated or Improper Forensic Science

    Since the late 1980s, DNA analysis has helped identify the guilty and exonerate the innocent nationwide. While DNA testing was developed through extensive scientific research at top academic centers, many other forensic techniques – such as hair microscopy, bite mark comparisons, firearm tool mark analysis and shoe print comparisons – have never been subjected to rigorous scientific evaluation. Meanwhile, forensics techniques that have been properly validated – such as serology, commonly known as blood typing – are sometimes improperly conducted or inaccurately conveyed in trial testimony. In some cases, forensic analysts have fabricated results or engaged in other misconduct.”

  13. windriven says:

    Thanks to both Allison Cummins and micheleinmichingan.

    Innocence Project claims on its home page to have helped exonerate 258 people wrongfully convicted. Good grief.

  14. Regarding the article and Harriet’s comment. I think it’s important to differentiate between instances when the science just isn’t there and shoddy craftmanship/user error.

    At this point it looks like from the article that genetic testing to predict risk factors and risk reducing measures just isn’t there yet.

    In these case of a person getting unexpected ethnic results from a genetic test, it’s hard for me to tell whether it’s incomplete science, user error, or good science that was unexpected. As I see it (having done a full 15 minute of wikipedia on genetic ethnic typing) one way to rule out user error would be to send the same test to multiple labs.

    I couldn’t tell from my 15 minutes whether the science is there yet.

    I can say that one of the more common test done for ethnic background has this breakdown of ethnicity.

    ” * Native American: Populations that migrated from Asia to inhabit North, South and Central America.
    * European: European, Middle Eastern and South Asian populations from the Indian subcontinent, including India, Pakistan and Sri Lanka.
    * East Asian: Japanese, Chinese, Mongolian, Korean, Southeast Asian and Pacific Islander populations, including populations native to the Philippines.
    * African: Populations from Sub-Saharan Africa such as Nigeria and Congo region.”

    Where’s Northern Africa? – I’m guessing it’s included in European.
    There’s also all the funny historical melting pot stuff, The Silk Road, New Orleans, The Caribbean, etc.

    Regarding Windriven’s comment ago whether a unexpected genetic outcome would effect someone self-perception. Intriguing. In our family it’s broadly understood that our great-great-grandmother (possible three greats..) was Native American. This seems to be an interal part of the identity on that side of the family. And actually, I’d be quite bummed out if I found it weren’t true. I’m sure I’d recover, be it is funny how important it seems.

    Of course this is no where near the impact of someone telling be that I’d almost certainly get breast cancer (without real evidence). They are completely different issues in my mind.

  15. stbloomfield says:

    @Harriet Hall, I thought it was only possible to say watch the ‘likely’ eye color is, and there are factors the effect the final outcome of eye color other than the dna (epigenetics?)

    As for saying what percentage of each “race” a person is, I think that all depends on the database the lab is using and what the makeup of it is.

    I keep seeing people making accusations about improper genetic claims but I don’t see much information as to why the claims are wrong. To simply say a black person somehow didn’t have any African dna isn’t enough. What are the reasons for that to happen? That sort of thing needs to be explained.

    We have lots of examples of how common information is misused: I know my blood type, and some people think that effects personality or diet, which is completely false. So I think its important that sites like this start educating people now. It isn’t enough to make accusations…tell me WHY.

    1. Harriet Hall says:

      @stbloomfield,
      Yes, most genetically directed traits can be modified by other factors. That applies to everything from eye color to diabetes. That’s one of the problems. Another BIG problem is that all these predictions are statistical ones based on population surveys. We don’t yet have a true understanding of what these genes actually do.

  16. stbloomfield says:

    Sorry, i don’t see an edit button:
    …possible to say WHAT the ‘likely’ eye color is…
    …factors THAT effect the final…

  17. Scott says:

    And why does the GAO not do a scientific study to determine if something is scientific?

    Because the burden of proof isn’t on them – it’s on those who want to sell the tests.

    What actually concerns me most about this subject is that some segments of the media (including some who normally are more reasonable) are portraying the GAO/FDA actions as an attempt to ban all direct-to-consumer genetic testing. Which it’s just not.

  18. Zetetic says:

    I can see it coming… Those companies with big vans that come in and do mass health screenings without any interperatations will be adding DNA testing soon!

Comments are closed.