This week, IBM's supercomputer Watson had quite a successful appearance in the man-versus-machine Jeopardy! showdown. Even with a couple flubs, Watson was able to handily beat two of the trivia game show's most prolific winners. Unfortunately, Watson's winnings won't make up for cancellation of IBM's hefty contract with the California Department of Education if the company doesn't meet deadlines to fix the state's data system. Maybe that's not so important to IBM, but it's not a trivial matter for California students.
We'll admit to being excited -- even thrilled -- by Watson's sheer computing power. However, that impressive display makes it even more frustrating to witness California's failure to get a fully functional education data system from IBM. That system should be able to answer fairly straightforward questions, such as:
* How many students who enter elementary school with limited English skills are still designated as English Language Learners when they arrive in middle school?
* Do eighth-grade students enrolled in Algebra 1 perform better, on average, if their teacher has a credential in math?
* Which California high schools graduate the highest proportion of young women who move on to major in computer science in California public universities?
The system should be able to follow students from kindergarten through high school graduation and beyond. It can't.
IBM has been beset with delays and technical complications in its contract with the state to create the California Longitudinal Pupil Achievement Data System, or CALPADS. The delays led Gov. Schwarzenegger last year to eliminate $6.8 million earmarked for the project. Of course, we don't know IBM's side. California's policy environment and historic disinterest in gathering good data might well contribute to delays. But, as we are fond of telling students, "No excuses."
A report out this week on states' capacity to collect data reveals that California compares poorly to other states. The Data Quality Campaign's sixth annual report reveals that half of the states are collecting the full 10 "essential elements" of data tracking. California was missing the ability to match student K-12 records with higher education.
Getting basic data is only an early step in a much longer process. Once CALPADS is in place, there are some difficult learning and political challenges. "States were looking at these 10 elements as a checklist and saying, "OK, we can collect these 10 things; we're done," Aimee Guidera, executive director of Data Quality Campaign, was quoted in Education Week. "We're saying, 'No, you're just beginning to be able to tap in and leverage the investments you've made.' "
Tapping into the full potential of data systems will require California to move beyond a narrow focus on outcomes data. Improving educational practice demands that we know more about the opportunities present in different schools and neighborhoods that lead to desired outcomes. That additional data must come from new sources, including students and educators, about the conditions that shape teaching and learning in their classrooms.
Even when IBM overcomes its technical difficulties for California, our data system will still be no Watson. Yet, just this one prototype machine has a lot to teach our practical-minded policymakers and communities. As stated on PBS' Inside Nova, "The significance of Watson goes beyond public perception... Watson isn't a single computer program, but a very large number of programs running simultaneously on different computers that communicate with each other."
Watson, in other words, isn't confined to preset programming of, for example, 10 conditions for this or that solution. To answer its questions, Watson seeks and communicates with new sources, penetrates the nuances of written and spoken language, and uses its power to arrive at trustworthy, best-bet answers.
Ultimately, the value of any super machine lies in whether humans can use it as a tool for problem solving and not confuse our basic tools with the solutions we seek. As IBM engineers complete California's longitudinal data system, California educators and community members need professional development and public engagement to access and reach beyond the technology and arrive at human decisions.
Follow John Rogers on Twitter: www.twitter.com/UCLA_IDEA
IBM Watson Wins Jeopardy, Humans Rally Back - PCWorld
IBM's Watson Faces Human Competitors on 'Jeopardy' Next Week ...
IBM's Watson Supercomputer Wins Practice Jeopardy Round ...
IBM's Watson supercomputer destroys all humans in Jeopardy ...
IBM DROPPED THE BALL ON THIS PROJECT.
Other companies could have had this done within a year.
For example, a recent (perhaps current?) president of the Michigan Education Association called for testing high school teachers on their content specialties. He wants to move the burden of proof of performance from a standardized test (with different test takers from year to year) to an evaluation of the teacher (the constant in that classroom). Undoubtedly there will be those who oppose this measure, but many I have spoken with in the MEA think it is a good step.
With respect to technology, more computers is not always the answer. In fact, computers are only as useful as the person at the helm makes them.
With respect to the method of teaching, I ask when was the last time you were in a classroom? (Your voice suggests you might be a teacher, but it is unclear.) The vast majority of teachers I know try to avoid the "chalk-talk/giver of knowledge" approach. Sometimes I think that we have moved too far from rote memorization in the early grades, particularly when it comes to math facts.
It isn't that simple.
IBM continues to work in partnership with the California Department of Education in order to meet the ambitious scope of this project. The project is in its final phase and we are on track to deliver remaining system functionality by the end of this school year. Clint Roswell, IBM