More

Featuring fresh takes and real-time analysis from HuffPost's signature lineup of contributors
Matthew Edlund, M.D.

Matthew Edlund, M.D.

GET UPDATES FROM Matthew Edlund, M.D.

What Science Stories Can You Really Believe?

Posted: 05/13/11 09:39 AM ET

Predictor -- or Marker?

Can you believe what you hear? Correlation versus causation is a major problem for media science stories. Recently I was asked in a TV interview about data that appeared to demonstrate earlier deaths for those who sleep more than eight hours a night.

The reason, I tried to explain, is that people who are sick will sleep longer than others. Rest is regeneration -- when you get sick, you need more rest for recovery. So you get more sleep.

Much of the data on "long sleepers" does not look closely at how underlyingly sick people are. It's like answering this question:

85 percent of Americans die in bed -- does that mean beds kill people?

I sincerely hope not -- I like what I do in bed. But people go into beds when they're sick, so they more commonly die there. This is a marker of outcome -- not a predictor. The cause lies elsewhere.

Similarly, there are many different sleep studies that have looked at oodles of people. One supposedly looked at more than a million and found that those who slept longer also died younger, with a "12 percent increase in mortality" for those sleeping more than 8 hours a night.

So Who Sleeps Long?

Some people do naturally. Yet, in general, people who sleep long are those with chronic medical or mental disease; sleep disorders, like sleep apnea or insomnia; or people with conditions like depression, which is known for causing both short and long sleep in its sufferers.

Can Sleeping Long Be a Problem Itself?

It's more important to check out what the problem might be that is causing people to sleep long. Over the years, I've seen many adolescents with ADD who slept 12 hours or more each night. If their ADD was treated with stimulants, they slept less -- but as soon as they were off the medications, they went back to patterns of very long sleep. A few did have narcolepsy, while others had mood disorders than can also cause long sleep.

It's important to look at people who normally sleep 7 to 8 hours a night and then suddenly find themselves sleeping far more. They generally deserve a proper medical work up.

There are rare prospective studies researching sleep duration. One of a small group, by Emeritus professor Dan Kripke of UC San Diego, found long sleepers did die earlier.

However, there are many biases in such studies. Not designed for long term follow up, they are filled with people with "volunteer bias" -- those who go into such studies can be very different from the general population or other clinical populations.

David Sackett, a famous epidemiologist, has described about 2000 forms of volunteer bias that may occur in almost any population study. A famous example came with the results of hormone replacement in the Nurse's Study from the 1980's.

The Nurse's Study was (and still is) a long term follow up of over 100,000 nurses. One of its first major findings was that nurses who used HRT after menopause decreased their risk of heart disease by 50 percent.

It seemed a clinical no brainer in the 1980's -- here's a way to cut the biggest killer of women in half -- just by adding hormone replacement post menopause.

The only problem was that the -- dead wrong. The group who took HRT turned out to be "first adopters" and health conscious people who concomitantly engaged in many health-promoting behaviors, many of which were not fully controlled in the study's design.

It's now clear that HRT is associated with an increased risk of heart disease. The message that HRT was helpful led to many needless deaths.

So with population studies, you need to do more than kick the tires. The mathematics used in "controlling" variables that change the results are complicated, and can be consciously and unconsciously manipulated by the biases of the researchers.

Bottom Line

Correlation may look like causation, but it isn't. Just as politicians make factual associations look like "real" connections, some results may associate with something else as the result of chance, or because there is a separate connecting event. Beds don't kill people, though terminally ill people will often stay in beds.

So be careful when you read headlines about new miracle cures and the wondrous effects of supplements. Study results can be manipulated readily, even when people are trying to do very honest, scientific work. And marketers know as well as anybody that in an age of quantitation, where better scores on standardized tests tell people which schools are "failing" or "improving," and where the "bottom line" in business counts more than anything (think BP), getting statistics to look right can go a long way in getting people to buy your product, or your political position.

It's better to have your own measure -- your private "BS" detector. Epidemiologists sometimes call it a "likelihood ratio." Likelihood ratios range from 0 (no way at all) to 1 (certainty). If you hear of a scientific finding which is completely opposite to everything you know, don't put a high likelihood ratio on it.

Chance alone may produce results that are highly unlikely in virtually any scientific study. But chance also favors the prepared mind -- especially yours.

 

Follow Matthew Edlund, M.D. on Twitter: www.twitter.com/therestdoctor

Predictor -- or Marker? Can you believe what you hear? Correlation versus causation is a major problem for media science stories. Recently I was asked in a TV interview about data that appeared to de...
Predictor -- or Marker? Can you believe what you hear? Correlation versus causation is a major problem for media science stories. Recently I was asked in a TV interview about data that appeared to de...
 
 
  • Comments
  • 96
  • Pending Comments
  • 0
  • View FAQ
Comments are closed for this entry
View All
Favorites
Recency  | 
Popularity
Page: 1 2 3  Next ›  Last »  (3 total)
photo
HUFFPOST SUPER USER
Vajara
vajara
02:12 PM on 05/15/2011
This is a very good article that can help us see through some of these investigative reports as well and certainly with all commercials and politicians. Truth is present all of the time; however, our minds are usually clouded with beliefs and lack of awareness about Nature and her Laws. Hmm, maybe I am missing the point or argument, but I also like going with my instincts when reviewing research articles as well as seeing who is paying for it?
photo
armywifee
From the Soviet Republic of Canuckistan
01:40 PM on 05/15/2011
Canadian scientists at the University of Alberta recently found a simple, inexpensive cure for many cancers.... It's a drug called dichloroacetate--commonly used in treating metabolic disorders, with no serious or lasting side effects.
This news has not been picked up by the media. So please spread the word:
http://hubpages.com/hub/Scientists_cure_cancer__but_no_one_takes_notice

The drug cannot be patented so Big Pharma is not interested. Treating cancer and cancer drugs are a billion $ industry each year. I suspect they're going to try very hard to discredit/squelch this.
05:17 PM on 06/09/2011
The study you mention was on mice and was not conclusive. Yes, DCA has been shown to have a moderate effect on some cancers, but it also may promote other types of tumor growth and has been linked to serious nerve damage in some patients. Just because something fits the "kill Big Pharma!" ideal does not necessarily make it a miracle drug.
photo
Aquest
No one here is exactly what they appear.
10:44 AM on 05/15/2011
It seems that if 85% of people die in bed, that there is a really good chance that you will die in bed. Hence, using a lot of people's logic, the way to live longer, if not forever, is to not sleep in a bed.
09:30 PM on 05/14/2011
Learn to read a scientific paper, then check the ones referenced by an article. If you can read a paper and understand proper scientific methodology, you'll be able to spot most bull news easily.
09:07 PM on 05/14/2011
I liked the Swedish study that showed women who drank 8 or more cups of coffee a day had fewer strokes.
HUFFPOST SUPER USER
Roxee
"Feeling" you're right, doesn't "prove" you are.
11:09 AM on 05/14/2011
First thing i do if the study is of interest is find out who funded it. I remember all those studies funded by big tobacco that "proved" smoking wasn't addictive. Also, if a study's findings have not been replicated approach with caution too.
photo
Jack Daniels Esq
Hold the ice
03:57 PM on 05/14/2011
The two most despised societies in the US - the AMA & ALA
photo
Iatros78
Science is the consensus of expert opinion
10:49 PM on 05/14/2011
The American Motorcyclist Association and the American Library Assocation? Really?
ThinkCreeps
Seriously, it's time.
10:22 AM on 05/14/2011
Which stories to believe? The ones where you understand the story and context in full.
Linda from Deerfield
Paying attention
10:10 AM on 05/14/2011
It's somewhat comforting to have an M.D. suggest that our B.S. detectors might be worthy. That is good advice in the financial arena, too, not to mention politics and a few other areas where there are money and power to be gained at our expense. Still, though, it is not that easy to hone your detector when, as an individual, you'll never have the resources and time to conduct the definitive study that proves your point of view once and for all. I think that it need not lead to paralysis, however -- it is part of the fun of this great experiment called life. Be brave, trust your instincts, and be willing to learn from it.
06:29 AM on 05/14/2011
It ALWAYS amazes me how people are so gulliable when it comes to studies/statistics. People don't realize how studies/statistics get skewed so easily.

With that being said, when it comes to sleep I have always been a long sleeper. My body just requires a good ten hours of sleep a night and I don't have any health problems at all. Where as a friend of mines only needs six hours of sleep a night and he didn't have any problems either.
photo
HUFFPOST SUPER USER
Susan Shaffer
tell me from the beginning
09:14 PM on 05/14/2011
or the fact that as you age your sleeping patterns change
i wonder what the demographics are of any of these sleep studies.
photo
french queen13
my beloved is mine and I am his
02:02 AM on 05/14/2011
There's also that favourite of the media, "Studies show that people who do X double their chances of dying!" Never mind that the chance of dying from doing X might have gone from one in a million to two in a million, or that everyone dies of something anyway ...
12:57 AM on 05/14/2011
When I want the truth, I may read government generated science, but I check elsewhere because the government seems to pay for so many "studies" that I think government gets what it pays for. In other words, if a trillion times the government pays to study cigarette smoke, the science lab will always find a new way to say "Cigarette smoke is no good for you".
But, I then think that maybe it is the same way with all government science. If the politicians have an agenda that I know nothing about, I can be misled.
I certainly wont believe a study that is all statistics, no matter who did the study, because it all depends on what information has been factored in. If one out of ten people get better after using a herbal cure, does that mean it works or doesn't? Even if it was nine out of ten who got better, I wont believe until I know exactly what actual science is involved in the cure.
photo
HUFFPOST SUPER USER
Tom Hendricks
see wikipedia
12:42 AM on 05/14/2011
The real question is WHY do we sleep. New ideas suggest it is the second half of a daily digestion cycle - you eat during the day, and then stop eating and work on absorbing nutrients in sleep - with a special emphasis on the digestion brain , ENS or enteric nervous system.
HUFFPOST SUPER USER
DrP
11:26 PM on 05/13/2011
I do not believe any science story that quotes observational studies that show an association between a certain behavior and a disease process. Association is not causation.
That is the premise of most of Gary Taubes's writings, which I do believe.
04:10 PM on 05/15/2011
Gary Taubes -- He the man!
photo
HUFFPOST SUPER USER
No death panels
There's no man with a trumpet. Only me.
08:57 PM on 05/13/2011
Wait, another lawyer against tort reform throwing a child's case in the mix but not mentioning the lawyers sucking 30-40% off the top of any award? Say it ain't so!
photo
Ytrus
''it's a map''
04:49 PM on 05/13/2011
Didn't you hear, jelly beans cause acne?

http://xkcd.com/882/