07/07/2010 05:12 am ET | Updated May 25, 2011

The Problem with Colorado SB10-191

In his most recent post, Alan Gottleib of EdNews Colorado writes --

In the past couple of months, however, I have sensed a shift in momentum on these big education questions. Until recently, people pushing school choice and the revamp of tenure and evaluation - the Obama-Duncan agenda - seemed to have the energy and mojo on their side.

Lately, though, the passion and commitment seems stronger among those fighting those reforms.

I would like to think that this shift in momentum means that those who believe we should be reforming our schools using system derived data are beginning to win out over the change-at-any-cost, let's-create-a-charter-school-for-no-reason education change advocates. For too long have the Obama-Duncan education change advocates painted those who want to use data to make reform decisions as "anti-reform" or pawns of the teachers' unions.

No one I have ever met believes our schools function properly. What most of the education advocates I know believe is, if you're going to make changes to the system, base those changes on strategies informed by data. Let's take SB10-191 as an example.

Everyone I know agrees that teacher evaluation needs to be improved, most would say dramatically. However, the mix of metrics used to perform this evaluation has to be developed to support the desired outcome of the system. (Sorry, creating educated kids isn't quantitative enough.) SB10-191 is not based on any sort of meaningful metrics development process. It proposes change for change's sake based on the opinions of disconnected legislators, albeit one of whom was a school principal.

If the representatives of the citizens of Colorado want to put into law that 50% of a teacher's evaluation should be based on the results of a standardized test, I, and many others, would like to know why not 2%? Why not 90%? Why not use scholastic benchmark tests alone? What about student grades? How about peer review? How should those various metrics be weighted and based on what criteria? (These are all good questions, but what about the one most interesting question: how does the state intend to pay for such an evaluation system?)

State Senator Michael Johnson and the proponents of SB10-191 have no answer to these questions, at least one that is supported by data. None of us has any idea if 50% of the teacher's effectiveness is reflected in students performance on CSAP. CSAP doesn't measure a 10th of what education should encompass, in my opinion. On the other side, an effective teacher should be able to teach to any standardized test. Why not make the value 90%? If you are an effective teacher, you should be able to get 90% of your students to perform better on the CSAP then they did last year, right?

The problem is this: everyone agrees that teachers should be held accountable for their performance, even teachers, but virtually no one has any idea how to measure that performance. I don't. You don't. If Michael Johnson knows, he's probably lying, unless he can produce some rigorously developed data to back him up. The public school systems and the Colorado Department of Education haven't done the work to support developing these metrics.

So, why not just start with some metrics? The first lesson of metrics development is, be careful what performance metrics you apply to your system. It is what you'll get out of the system. Systems, whether intentionally or not, will perform to meet the quality metrics applied to them.

Take a truck, for example. If the quality measurement is, Does the door open and close, 98% of the doors will meet this requirement. A significantly less number will be able to open and close over the life of the truck. The system is not measuring that parameter. (Sound like Detroit to you?)

So, my opposition to SB10-191 is based on the above case study. We have no idea what we should be measuring to determine a teacher's performance.
  • We have no idea what our schools' responsibilities are. For example, do we, as a society, expect our schools to just teach our kids how to add in 1st grade? Do we expect that to be taught along with an understanding of how addition is useful in the world around us? How about the apparent antithetical nature between addition and subtraction? What about the fact that there is no such thing as subtraction -- it is all addition and the model must sometimes account for negative numbers? What about the ethical ramifications of using addition in a subtraction-based society? (Don't believe me? Look at "education reform," as the Obam-Duncan crowd like to call it -- all we talk about is getting rid of bad teachers, not how to create good teachers through improving the collegiate training process. That would be addition.)
  • Is the school responsible for a student's character education or is that something we are leaving to parents? Is it realistic to think that parents can handle character education by themselves? How do we address socio-cultural development? What about one-to-one and group-to-individual negotiation?
  • What are the teacher's responsibilities in terms of parent interaction and requirements fulfillment? From my experience, more teachers have been removed from the classroom because they did not meet parents' untrained expectations about education than for actually failing to teach to the Colorado state standards.

What CEA/NEA are saying is simple: Let's make sure we understand what is expected of the system before we start measuring what we are getting. It's pretty logical. We can then use teacher assessment data to judge how the system is performing in order to meet the system's requirements. Using that information, we can make changes to the system as necessary.

In my one-time profession, we liked to call this approach data-driven enterprise transformation. Look it up, Michael Johnson, Alan Gottleib, and superintendents from around the state. It might actually get the state of Colorado, especially DPS, somewhere other than flat CSAP scores.