DTC Snps... no more risk factors!
I've been reading Daniel's blog again. Whenever I end up commenting on things I don't understand well, that's usually why. Still, it's always food for thought.
First of all, has anyone quantified the actual error rate on these tests? We know they have all sorts of mistakes going on. (This one was recently in the news, and yes, unlike Wikipedia, Daniel is a valid reference source for anything genomics related.) I'll come back to this point in a minute.
As I understand it, the risk factor is an adjustment made to the likelihood of the general population in characterizing the risk of an individual suffering from a particular disease.
So, as I interpret it, you take whatever your likelihood of having that disease was multiplied by the risk factor. For instance with a disease like Jervell and Lange-Nielsen Syndrome, 6 of every 1 Million people suffer from it's effects (although this is a bad example since you would have discovered it in childhood, but ignoring that for the moment we can assume another rare disease with a similar rate.) If our DTC test shows we have a 1.17 risk factor because we have a SNP, we would multiply that by 1.17.
6/1,000,000 x 1.17 = 7/1,000,000
if I've understood it all correctly, that means you've gone from knowing you have a 0.000,6% chance to being certain you have a 0.000,7% chance of suffering from your selected disease. (What a great way to spend your money!)
But lets not stop there. Lets ask about the the error rate on actually calling that snp is. From my own experience in SNP validation, I'd make a guess that the validation rate is close to 80-90%. Lets even be generous and take the high end. Thus:
You've gone from 100% knowing you've got a 0.000,6% chance of having a disease to being 90% sure you have a 0.000,7% chance of having a disease and a 10% sure you've still got a 0.000,6% of having the disease.
Wow, I'm feeling enlightened.
Lets do the same for something like Celiacs disease, which is estimated to strike 1/250 people, but is only diagnosed 1/4,700 people in the U.S.A. - and lets be generous and assume that the SNP in your DTC test has a 1.1 risk factor. (Celiacs is far from a rare disease, I might add.)
As a member of the average U.S. population, you had a 0.4% chance of having the disease, but a 0.02% chance of being diagnosed with it. That's a pretty big disparity, so maybe there's a good reason to have this test done. As a Canadian it's somewhat different odds, but lets carry on with the calculations anyhow.
lets say you do the test and find out you have a 1.1 times risk factor of having the disease. omg scary!
Wait, lets not freak out yet. That sounds bad, but we haven't finished the calculations.
Your test has the SNP.... 1.1 x 1/250 = 0.44% likelihood you have the disease. Because Celiacs disease requires a biopsy to definitively diagnose it (and treatment does not start till you've done the diagnosis), would you run out and submit yourself to a biopsy on a 0.44% chance you have a disease? Probably not unless you have some other knowledge that you're likely to have this disease already.
Then, we factor in the 90% likelyhood of getting the SNP call correct: You have a 90% likelihood of having a 0.44% chance of having the disease, and a 10% likelihood of having a 0.4% chance of having the disease.
Ok, I'd be done panic-ing about now. And we've only considered two simple things here. Lets add one more just for fun.
lets pretend that an unknown environmental stressor is actually involved in triggering the condition, which would explain why the odds are somewhat different in Canada. Since we know nothing about that environmental trigger, we can't even project odds of coming in contact with it. Who knows what effect that plays with the SNP you know about.
By now, I can't help thinking that all of this is just a wild goose chase.
So, when people start talking about how you have to take your DTC results to a Genetic Counsellor or to your MD I really have to wonder. I can't help but to think that unless you have a very good reason to suspect a disease or if you have some form of a priori knowledge, this whole thing is generally a waste. Your Genetic Counsellor will probably just laugh at you, and your MD will order a lot of unnecessary tests - which of those sounds productive?
Let me make a proposal (and I'm happy to hear dissent): Risk factors are great - but are absolutlely useless when it comes to discussing how genetic factors affect you. Lets leave the risk factors to the people writing the studies and ask the DTC companies to make a statement: what are your odds of being affected by a given condition? And, if you can't make a helpful prediction (aka, a diagnostic test), maybe you shouldn't be selling it as a test.
First of all, has anyone quantified the actual error rate on these tests? We know they have all sorts of mistakes going on. (This one was recently in the news, and yes, unlike Wikipedia, Daniel is a valid reference source for anything genomics related.) I'll come back to this point in a minute.
As I understand it, the risk factor is an adjustment made to the likelihood of the general population in characterizing the risk of an individual suffering from a particular disease.
So, as I interpret it, you take whatever your likelihood of having that disease was multiplied by the risk factor. For instance with a disease like Jervell and Lange-Nielsen Syndrome, 6 of every 1 Million people suffer from it's effects (although this is a bad example since you would have discovered it in childhood, but ignoring that for the moment we can assume another rare disease with a similar rate.) If our DTC test shows we have a 1.17 risk factor because we have a SNP, we would multiply that by 1.17.
6/1,000,000 x 1.17 = 7/1,000,000
if I've understood it all correctly, that means you've gone from knowing you have a 0.000,6% chance to being certain you have a 0.000,7% chance of suffering from your selected disease. (What a great way to spend your money!)
But lets not stop there. Lets ask about the the error rate on actually calling that snp is. From my own experience in SNP validation, I'd make a guess that the validation rate is close to 80-90%. Lets even be generous and take the high end. Thus:
You've gone from 100% knowing you've got a 0.000,6% chance of having a disease to being 90% sure you have a 0.000,7% chance of having a disease and a 10% sure you've still got a 0.000,6% of having the disease.
Wow, I'm feeling enlightened.
Lets do the same for something like Celiacs disease, which is estimated to strike 1/250 people, but is only diagnosed 1/4,700 people in the U.S.A. - and lets be generous and assume that the SNP in your DTC test has a 1.1 risk factor. (Celiacs is far from a rare disease, I might add.)
As a member of the average U.S. population, you had a 0.4% chance of having the disease, but a 0.02% chance of being diagnosed with it. That's a pretty big disparity, so maybe there's a good reason to have this test done. As a Canadian it's somewhat different odds, but lets carry on with the calculations anyhow.
lets say you do the test and find out you have a 1.1 times risk factor of having the disease. omg scary!
Wait, lets not freak out yet. That sounds bad, but we haven't finished the calculations.
Your test has the SNP.... 1.1 x 1/250 = 0.44% likelihood you have the disease. Because Celiacs disease requires a biopsy to definitively diagnose it (and treatment does not start till you've done the diagnosis), would you run out and submit yourself to a biopsy on a 0.44% chance you have a disease? Probably not unless you have some other knowledge that you're likely to have this disease already.
Then, we factor in the 90% likelyhood of getting the SNP call correct: You have a 90% likelihood of having a 0.44% chance of having the disease, and a 10% likelihood of having a 0.4% chance of having the disease.
Ok, I'd be done panic-ing about now. And we've only considered two simple things here. Lets add one more just for fun.
lets pretend that an unknown environmental stressor is actually involved in triggering the condition, which would explain why the odds are somewhat different in Canada. Since we know nothing about that environmental trigger, we can't even project odds of coming in contact with it. Who knows what effect that plays with the SNP you know about.
By now, I can't help thinking that all of this is just a wild goose chase.
So, when people start talking about how you have to take your DTC results to a Genetic Counsellor or to your MD I really have to wonder. I can't help but to think that unless you have a very good reason to suspect a disease or if you have some form of a priori knowledge, this whole thing is generally a waste. Your Genetic Counsellor will probably just laugh at you, and your MD will order a lot of unnecessary tests - which of those sounds productive?
Let me make a proposal (and I'm happy to hear dissent): Risk factors are great - but are absolutlely useless when it comes to discussing how genetic factors affect you. Lets leave the risk factors to the people writing the studies and ask the DTC companies to make a statement: what are your odds of being affected by a given condition? And, if you can't make a helpful prediction (aka, a diagnostic test), maybe you shouldn't be selling it as a test.
Labels: biology, General Musings, Off topic, SNPS
6 Comments:
2 things:
1. Celiac. (sorry)
2. The CBC ideas podcast had a series a couple weeks ago on overuse of diagnostic tests. It puts this recent DTC testing stuff in an interesting light.
Thanks for the corrections. There were plenty of other typos too... DOH!
I'll have to take a look to see if i can track down the CBC podcast - definitely sounds interesting!
I don't have time to go into all of the many errors in this post, but for a start, the odds ratio associated with the celiac snp is about 5-10X *per allele* (about 50X for a homozygote). This HLA allele accounts for about 90% of celiac disease and its mode of action is well understood.
I understand this is just a blog and you are not supposed to be an expert, but you should do some basic reading on genetics before posting misinformation. Or better yet, leave this stuff to Daniel MacArthur.
Hi Steve,
That's a really uninspirational way to comment on my blog.
Fist, Thank you for pointing out that the Celiac SNP is actually much higher than I'd estimated. I had picked the number out of thin air (as I'd mentioned in the post.)
Second, If you'd like to take the time to point out errors, I'd be happy to fix them. Perhaps in the meantime you should leave the commenting to Daniel, as he is happy to educate instead of belittle. Or maybe you ought to do some reading on polite communication?
Yo Antony,
You are right that the background rates of a disease are very important, and completely change the intepretation of genetic risk factors. I wrote a blog post recently putting some actual numbers on a person's genetic risk for various diseases of different rarity:
http://www.genetic-inference.co.uk/blog/?p=544
Take home message - you can get more information than you might think, though nowhere near as much as you might want. And, the entire 'feel' of genetic information changes with the disease prevalence.
A few notes on your post.
- GWAS sites tend to be well behaved, known polymorpisms - if a chip calls it, it has a very high chance of getting it right (>99%).
- You can easily take any uncertainty into account when you give risk factors. Your example of the 90% uncertain celiac SNP is entirely equivalent to saying that you have a 0.436% chance of having the disease; the uncertainty in the call is no different from the uncertainty in the risk data, and can be combined normally.
Finally catching up on my long-neglected RSS feed...
Following up on Luke's point about the accuracy of these scans: based on comparisons between scans from two different companies on the same person the error rate is on the order of 1 in every 10,000 sites.
Post a Comment
<< Home