THE BLOG
01/05/2011 06:17 pm ET Updated May 25, 2011

Evaluating Student Learning: The Moneyball Approach

Previously I have written about what a pathetic mess is made of education reform when a group of MBAs, with little understanding of education, begin to call the shots in education policy. They invoke a kind of faith-based reliance on so-called "market forces." And pretty soon we are caught up in a Hobbesian nightmare, a war of all against all.

But, in thinking about my diatribe about the MBAs, I have decided to qualify the argument a bit. Because it is not that MBAs must be blinkered and ignorant. It is just that some of them are. This occurred to me while watching the baseball playoffs and thinking about Moneyball, Michael Lewis' brilliant exploration of Billy Beane and his strategy for turning around the Oakland A's even as a small market, low-budget team. Beane's management team was decried by the baseball traditionalists as a group of pinheads and bean-counters -- basically MBAs.

But what they came up with was extraordinary. Instead of using the traditional narrow measurements of a player's value, such as batting average or runs batted in, they examined the many ways a player can add value to a team -- the slugging percentage (which accounts for multiple base hits), on-base percentage (which includes times one was walked or hit by a pitch), etc. Moving away from narrow measures, they did a more complex analysis. As assessment data was more democratized and as more people could use computers to broaden their analysis, these small teams could go around the traditional hierarchies that had dominated player hiring and promotion. And, using their more complex analytical system, the Oakland A's were able to recruit AAA players who were low cost (maybe their batting average was not stellar) but high value to the team.

But here's the take home point. The upstart MBAs in the Billy Beane mold were broadening their metrics. They were choosing more and more data points, more information, more nuanced understanding of performance, to reward value. But our education MBA's have taken the lazy route. They should be broadening assessments to understand what students know and are able to do -- looking at qualitative evaluations, performance and portfolio and project based assessments, and learning in multiple modes that include creative and arts fields. Instead, they have narrowed the assessment -- really to only one measure, the standardized test. And that way lies disaster. Because, as anyone in business can tell you, a single metric bends all the efforts to polishing up that one measure, to the detriment of other important factors and even to the derailing of the whole enterprise.

We can still get off this train. We can still reject the willed ignorance of the testing mania that has settled on the whole educational world. But we had better do it quickly -- it's doing irreparable damage to students and communities.