THIS BLOG POST WAS WRITTEN BY TWO GRADUATE STUDENTS AND TWO POSTDOCTORAL FELLOWS AT THE UNIVERSITY OF DELAWARE: CARA CUCCUINI-HARMON (GS), REBECCA A. DORE (PF), ALEXIS GEIBLER (GS), AND ILYSE RESNICK (PF).
Information is exploding. Claims made online by articles and advertisements may or may not be true. How are we to know? Consider, for example, the Your Baby Can Read program. The creators claimed that infants who used this program would read novels by the time they were toddlers - and hopeful parents believed! A savvy consumer of information needs to ask whether these claims are backed by good science. Dr. Susan Neuman, an expert on reading at New York University, tested the Your Baby Can Read program. She found that after 7 months of using the program, toddlers were not any better in reading or pre-reading skills compared to children who had not had used the program. Even before this evidence was published, and based on the evaluations of experts in the field, the Campaign for a Commercial-Free Childhood filed a complaint with the Federal Trade Commission, who ruled against the Your Baby Can Read company for false advertising. Here, we use some of the advertising claims made by the Your Baby Can Read program to demonstrate how to use three red flags for evaluating scientific claims in popular media.
Red Flag #1: Don't play with my heart!
The Your Baby Can Read program made statements like, "Every parent can seize this small window of opportunity and truly give their child an early start on all learning." Such emotive language tries to create a sense of urgency as it preys on parents' obligation to do what is best for their child. Using emotional tactics - our first red flag - attempts to hit parents right in the gut and likely causes anxiety. Good articles are written in a straightforward way and present both sides of the argument; they don't play with readers' feelings.
Red Flag #2: Show me the methods
Credible science describes how a study was done and what was found, rather than providing only a claim that lends itself to a catchy headline. The Your Baby Can Read program doesn't appear to cite any studies about their program's effectiveness - a huge red flag! Even when this information is provided, it may not be the kind of evidence that unequivocally supports the claim. For example, imagine a study in which children whose parents bought the Your Baby Can Read program really did have better reading scores than children whose parents did not purchase the program. It is possible that something unrelated to the materials may have led to this difference in reading ability. Parents who sought out a reading program for infants might also be more likely to do other, related activities such as many hours of storybook reading together.
Red Flag #3: The plural of anecdote is not data
What type of evidence is provided? A bright red flag is waving when authors only give testimonials or anecdotes to back their claims. Keep in mind that advertisers or authors choose which anecdotes to include and may select just the ones that support their argument. Unfortunately, these anecdotes do not mean the product worked well for the majority who bought it. While the Your Baby Can Read Program had loads of testimonials, we want to see real data on how well the program worked. Neuman did that. Importantly too, she had no vested interest in the outcome of her study. Studies like Dr. Neuman's are often available on the web. Anecdotes should be used only to help describe what the data show.
Reader beware: There are red flags everywhere on the internet!
It can be hard to separate fact from fiction. Check the facts to avoid falling prey to false claims. Consider the wording, the methods, and the evidence before believing. Red flags likely mean fiction is winning over fact.