The national and international press had fun with the cell phone text messaging experiment that was conducted in Oklahoma City middle schools. Molly Mulghire's "Surprise! Inspirational Text Messages Won't Improve Teens' Grades" in the New York Observer was preceded by the heading, "Go Home Science You're Drunk." Its subtitle was "But They Are Usually So Obedient!" Mulghire's take on the effort to provide free cell phones and minutes, and inspirational texts was, "Kids were bribed with free cell phones in return for receiving daily texts encouraging them to study hard, stay in school, blah blah blah."
In England, the Guardian reported, "A groundbreaking experiment that bombarded US high school students with inspiring text messages was found to be a success on all counts except one: it made no difference to how the students performed in school."
The post-mortem on the latest failed effort by non-educators to use carrots and sticks to improve education stands in stark contrast to the official enthusiasm about Oklahoma City's 2010 program where 1,870 middle school students received the free phones so they could earn cell phone minutes for reading books as a part of a Harvard University economist's research into academic motivation. Two groups of those students also received text messages crafted by an award-winning public relations company.
The program was the brainchild of MacArthur Genius Grant recipient Roland Fryer. Three years ago, the program was spun as a method of increasing student performance, as opposed to discovering whether a text messaging campaign could get students to say that they value education.
When announcing the experiment, the OKCPS superintendent Karl Springer said, "This is a cutting-edge kind of activity ... We believe it will be successful. If we can increase literacy here, why not also in Detroit, Boston, Knoxville? This could have a game-changing effect on our students."
I doubt many teachers or administrators (if they were off the record) thought that the experiment ever had a chance of success. It had already failed in New York City. At the start of the program, the Daily Oklahoman reported, "Edward Deci, a psychology professor at the University of Rochester in New York and an expert on motivation, said Fryer's previous experiments so far have 'shown very little positive effect' and said that 'incentives don't help improve performance by and large' over the long-term."
Now, Fryer finds that students said that they gained a new appreciation of school, so their "beliefs changed." Because they said they worked harder, "effort seemingly increased." But, "there were no tangible benefits."
The economist's current conclusion is that the study is a contribution to understanding "the shape of the production technology around the pre-treatment equilibrium." It shows that "students do not know the education production function and thus are not sophisticated enough to translate effort into measurable output." It also concludes, "students in Oklahoma Public Schools do not have accurate knowledge of the returns to schooling."
It would be easy to join the ridicule of Fryer's study, but I am more concerned about the accuracy of his knowledge of real-life schools and wonder whether it explains a consistent flaw in the economist's methodology. My doubts go beyond Fryer's spinning of survey data and maintaining that teens worked harder because they said that they did. The bigger purpose of schooling is teaching kids to work smarter and the incentives did not do so.
Real world, what would we say about a teacher whose students learned that "male high school dropouts go to prison four times more often than men who went to college" and that "college graduates make 54 percent more money than college dropouts?"
Yes, we would say, the teacher conveyed more knowledge. But, how would we judge that instruction if those students could not use that knowledge to infer the answer to the multiple choice question, "15.5 percent of high school dropouts are unemployed. What percentage of college graduates are unemployed?"
According to Fryer's perspective, text messages did not inform students of the answer to that placebo question, so text messages conveyed more knowledge. He spins the inability of students to figure out the answer as proof that the placebo effect did not occur. He forgets, however, that the purpose of schooling is teaching children to learn how to learn. Fryer ignores his own evidence that such sort of learning did not occur.
Rightly or wrongly, I infer that Roland Fryer will continue craft sophisticated models to test whether incentives and disincentives will make schools more accountable, will transfer more knowledge and, someday over the rainbow, will result in more real learning. The question is whether he will continue to spin findings that undercut his hypotheses.