Student Test Scores Should Be Used To Rate Teachers In Teams: Study
WASHINGTON -- Standardized tests should rank students by percentile and rate teachers in teams, according to a new policy brief by Derek Neal, an economics professor at the University of Chicago.
"I'm very opposed to ever using this [data] to give individual scores for teachers," said Neal, speaking at a Tuesday conference hosted by the Brookings Institution's Hamilton Project.
Educational research like Neal's is appearing as standardized tests have become more important to school funding decisions and play a larger role in the evaluation, hiring and firing of teachers. At least 26 states now mandate teacher reviews that take standardized testing into account. Many education reformers stress the use of data to rate teachers -- but, as Neal noted, these exams are often imperfect. Critics of this development argue that increased focus on tests won't improve student learning if the tests aren’t measuring the right things.
Neal's dissatisfaction with standardized exams derives from their dual use. "You have a test that's being used to measure how the students are performing in a system over time," Neal said. "At the same time, you're taking those test scores and creating performance metrics for the educators."
The two interests undermine each other, Neal said. The characteristics that make tests good at measuring achievement over time -- such as consistent formatting -- also make it easier for teachers to teach to the test, which corrupts the tests' usefulness in measuring the adults' performance in the classroom.
According to Neal, students often perform much higher on standardized tests that are used in teacher accountability systems than on tests that are not, suggesting that the structure leads teachers to focus on test-specific skills. "The predictability that makes consistent scaling possible in theory invites ... coaching that contaminates the scale in practice," he said.
"Race to the Top is not going to get you around this," said Neal, referring to the federal educational funding competition that inspired many state changes and is financing the development of new exams.
Instead of linking scores to individual teachers, Neal suggested calculating student scores on a percentile basis statewide and holding teachers accountable in groups -- all those within one school in the same grade and subject. "You take that number [the percentile score], average it over all the kids in a grade or school -- that's a winning percentage for that fifth-grade math team," he said. These scores create a performance curve, he said, and offer a ready means for identifying failing schools by how they perform against each other.
Neal presented his research at the Brookings Institution conference, "Promoting K-12 Education to Advance Student Achievement," which examined the role incentives play in education.
States use exams to determine whether students are proficient in a certain subject, a measuring encouraged by the No Child Left Behind mandates. Neal noted there is no consensus on what the dividing line, known as the cut score, should fall between students ranked as proficient and failing. The state-by-state setting of cut scores, he said, allows states to game their numbers: New York state, for example, was revealed to have lowered its cut score when it was discovered that a large number of students were scoring right above the proficiency line.
"If you try to set targets, those targets can be manipulated," said Neal, advocating instead for relative performance rankings.
Also speaking at the conference, Robert Hughes, president of New Visions for Public Schools, noted that relative performance rankings can be tricky because it's difficult to figure out which student factors to control for and how. "Getting that peer-to-peer analysis is very complicated," he said.
Michael Mulgrew, president of the United Federation of Teachers, said teacher accountability systems should encourage professional development. He called for a "fast process for removing teachers who are not doing their jobs well."
Sometimes the push for greater accountability can go too far, said News Corp. executive Peter Gorman, who until recently headed the Charlotte-Mecklenburg public school system in North Carolina. "I made a horrible mistake," he said. "I came out too quick on value-added."
Gorman was referring to the ranking of individual teachers through a formula that calculates how much teachers advanced their students' achievements relative to expected growth. He found that students grew the most when teachers worked together, the very collaboration factor Neal's system would measure.
But when Mulgrew noted that New York City teachers had not routinely been given approved curricula -- he said he had to introduce a provision guaranteeing that in a recent bill, and now "we have a lot of people breaking the law" -- Neal was taken aback.
"I just assumed that every school had a big curriculum book from the state," Neal said. The audience laughed. He had based his model on this assumption.
After the event, Mulgrew said he was surprised about the gap between Neal's expectations and reality. "The assumption that all schools have curricula and development teams, that's something I've run into before," Mulgrew said. "It would be helpful if academia was working with school systems to understand these things."
Loading comments… 
First Posted: 09/27/2011 7:30 pm Updated: 11/27/2011 4:12 am