NEWS
04/03/2018 05:45 am ET Updated Apr 03, 2018

Home Genetic Tests May Be Riddled With Errors, And Companies Aren't Keeping Track

Before you sign up for a home DNA test, learn more from this couple's story.
Julie Kennerly-Shah and Summit Shah.
Courtesy of Julie Kennerly-Shah and Summit Shah.
Julie Kennerly-Shah and Summit Shah.

Julie Kennerly-Shah and Summit Shah were dating back in 2016 when they decided to buy kits from 23andMe, the popular consumer DNA health and ancestry company. Like many customers, they were doing it to learn more about their heritage, but also to see if there were any health insights they could act on.

When they got their results a few weeks later, they mostly used the reports to tease each other about who was genetically more likely to have body odor or wet earwax.  

“It was kind of fun to joke around with each other and say that I don’t need to shower, or you need to shower more frequently,” said Shah.

They also exported their data to Promethease, a third-party DNA analysis company that links a person’s DNA report to published research on their particular genes, summarizing the potential medical implications.

Kennerly-Shah, a pharmacist, and Shah, a doctor, were in fact more eager to see what results Promethease would turn up. As medical professionals, they felt equipped to parse the potentially complicated reports that would turn up. The initial reports weren’t very interesting, but the site promised to provide new insights over time as they added more research to their database. 

In January 2018, the couple, now newlyweds, logged back into Promethease. They were shocked to see a new notification on Kennerley-Shah’s dashboard: She had a gene mutation linked to Lynch syndrome, a rare genetic condition that gives people an over 80 percent risk of developing colon cancer and women a 71 percent risk of endometrial cancer.

She also had a higher risk of familial hypertrophic cardiomyopathy, which is a genetic heart disease that increases the risk of arrhythmias and sudden death, even in people with no symptoms. 

The couple was terrified.

While both conditions are serious, they were most concerned about Lynch syndrome. Kennerly-Shah was very familiar with this devastating genetic condition — she worked at Ohio State University’s Comprehensive Cancer Center, which is currently doing a genetic study with colon cancer patients and their families to see if families with multiple cases of colon cancer may have the condition.

“Summit was probably more scared than I was, and I think that probably increased my anxiety,” said Kennerly-Shah.

The couple began to make new plans. They might have to start a family earlier than they wanted ― hysterectomies are often recommended as a preventive measure for women with Lynch syndrome. And they’d likely have to achieve pregnancy through IVF treatments, which could ensure they only implanted embryos that did not carry the mutation.

This story has a happy, if somewhat nerve-wracking ending: Acting on the advice of colleagues, Kennerly-Shah scheduled an appointment with a genetic counselor at the Ohio State University Wexner Medical Center to confirm that she was at risk for Lynch syndrome.

When the results came back, Kennerly-Shah was in the clear. She did not have the genetic mutations linked to either condition. Promethease had turned up a false positive result. 

In one study, 40 percent of direct-to-consumer results were actually false positives.

23andMe, the only direct-to-consumer genetic test authorized by the FDA to offer reports on genetic risk, claims to have more than 2 million customers. Only a fraction of these customers upload their genetic report to Promethease, whose founders said in a past interview that the site generated up to 500 reports a day. And there is no data on how many customers take action based on what they find. 

But as more people seek personal genetic information to help with their medical decision-making, experts are growing concerned over some of the shortcomings of consumer-facing genetic data companies that clinical, professional-grade services do not have. 

Promethease does not share information with its customers on their rate of false positive results. In fact, neither does 23andMe.

But scientists at Ambry Genetics, a clinical diagnostics company, found that 40 percent of 49 direct-to-consumer reports they reanalyzed using clinical grade lab testing were actually false positives. (A spokesman for 23andMe noted, however, that the genetic variants Ambry ran from 23andMe were accurate.)

There’s also the concern over how patients will act on the information they receive. People who get results like Kennerly-Shah’s may not know to confirm them in a clinical setting before seeking treatment, for example. 

A doctor would have never ordered a cancer or heart disease genetic screening panel for someone like Kennerly-Shah, who had no symptoms and no family history of disease, if it hadn’t been for her Promethease results.

“If their doctor wanted to order this test, they couldn’t do it,” said Dr. Ellen Wright Clayton, a genomics and ethics researcher at Vanderbilt University who was not involved in Kennerly-Shah’s care. “But the question that you could ask is, if this is a test that we don’t think is medically appropriate in general practice, why should …  the healthcare system have to pay for a follow up? That’s part of what we’re talking about here.”

In general, these direct-to-consumer DNA test results are so compelling that they can’t reasonably be ignored by conscientious doctors, so they typically set off a cascade of medical interventions — such as additional testing — with no guarantee that health insurance coverage will pay for anything. Because of the results from Promethease, for example, Kennerly-Shah had to pay $250 out of pocket for in-depth testing to confirm the results.

And that’s a best-case scenario. Less research-driven doctors could recommend invasive preventive treatments that would have been completely unnecessary.

Dr. Amanda Toland, a cancer and genetics researcher and member of the care team that reviewed Kennerly-Shah’s case, is concerned that a more aggressive doctor, or a doctor who isn’t well-versed in genetic research, could have recommended a hysterectomy or similarly invasive medical treatment simply based on the results of a patient’s direct-to-consumer analysis. 

For instance, when doctors were first learning how mutations on the genes BRCA 1 and BRCA 2 might contribute to higher risk of breast and ovarian cancer, she did hear of women getting preventive mastectomies even though their genetic results were inconclusive or showed no association with disease.

Doctors could say, “OK, if you have this mutation, you can think about having your ovaries out and we need to get you doing these screenings,” Toland said. “You could have complications from any types of those things. It’s scary. It’s very scary.”

The Problem Of False Positives 

Anecdotally, doctors are confirming the prevalence of false positives in patients who come to their office with test results in hand. For example, Toland said that in addition to Kennerly-Shah, her center saw two other patients trying to confirm serious disease risks they had found on direct-to-consumer genetic testing that month. All three of them turned out to be false positives.

When asked about this issue, the commercial facilities are quick to recast their role as informational rather than clinical ― a departure from the Silicon Valley premise that more data means more actionable insight.

A spokesman for 23andMe said the company doesn’t recommend or endorse sending your genetic data to third-party sites for health analysis, as the company’s research on raw genotype data has not been completely validated for accuracy. However, 23andMe makes the data available for export because the company believes that people have a right to their own genetic data.

Greg Lennon, a spokesman for Promethease, called the company a “literature retrieval service” and emphasized that it doesn’t generate or edit any data that consumers upload from 23andMe.  

This kind of “caveat emptor” language is par for the course for companies that want to minimize their liability. But companies don’t sufficiently warn consumers ― patients, really ― about the expensive and emotional “wild goose chase” that these test results can trigger, says Clayton.

In a conversation with Shah about false positives on a Promethease-moderated Reddit thread, an unnamed company spokesperson denied responsibility for this issue.

“Most of these DTC test results are not clinically validated, and while the data from them can be quite informative, it’s a caveat emptor world, and before a user got their data, they checked quite a few online ‘boxes’ acknowledging they understood this.”

But the company went on to say they were collecting data on errors.

“We are indeed collecting data on false positives (and false negatives) and sharing it when it’s reliable,” the spokesman told Shah.

  • Courtesy Summit Shah

The Shahs are no longer thinking about starting a family as soon as possible. But they do hope that people learn from their experience.

“Our concern is, how do we get this information out to the general public who may be getting ready to experience something similar?” Kennerly said. “So that when they do that Google search that we did in the first couple minutes of seeing those results, they have a resource available and they know what next steps they should take.”

CONVERSATIONS