Florida Teacher Evaluations Tied To Student Test Scores
A committee of administrators, teachers, union leaders and parents from around the state met to decide which variables to include in the formula. Copa said they decided on a more complex model to gain more reliability and accuracy.
To try to isolate the teacher effect, the model weighs 10 factors, like student attendance and disability status.
Each factor has a different weight for each subject and grade level, explained Jon Cohen, executive vice president and program director for assessment at the American Institutes for Research.
The strongest factor in predicting a student's future score--explaining about 60 percent of the variance in achievement--is that student's prior test score. "If I know how you did on the math test last year and you did well, we'd expect you to do well this year," Cohen said.
POVERTY LEFT OUT
The formula does not take into account a student's race, gender or socioeconomic status--despite evidence that such characteristics are linked to student achievement.
Florida legislators opted to leave poverty out of the equation.
"They said every child should be able to learn, regardless of their poverty level," said Gisela Feild, Miami-Dade's administrative director for assessment, research and data analysis. Otherwise, it would have been good to include, Feild said. "We all know that poverty is a big indicator."
In Miami-Dade, just over 60 percent of the 343,500 public-school students qualify for a free lunch in the National School Lunch program. In Broward, half of the 234,000 students are eligible.
Cohen said that if poverty influences a student's score one year, it likely impacted the previous year's score, too. "It's pretty much embedded in the child's scores when you control for prior achievement."
Maxwell, who served on the committee, worries that with no weight given to schools in low-income neighborhoods, teachers may leave for charter or private schools.
"I don't think you want to have a system that creates enclaves, where the best teachers leave a school based on that student population having poor performance," Maxwell said.
RELIABLE AND COMPLEX
Karen Aronowitz, president of United Teachers of Dade, said the value-added formula is not a fair instrument and discourages collaboration among teachers. "It's the wrong model and people know it."
Parents also worry about how their children's teachers will be evaluated. Jeanne Jusevic, a parent activist in Broward County, said, "We're taking something that should be a diagnostic tool to let the school know where the kids are, and we're making it the end-all, be-all."
In one study, Harris and other researchers asked principals to rate specific teachers. They found that the results generally corresponded with those teachers' value-added scores. But in several cases, a teacher deemed wonderful by a principal fared poorly when measured by student test-scores.
In another study, the error rate was calculated to be 25 percent, based on three years of data, according to a 2010 study by Mathematica Policy Research--meaning the model would incorrectly rate 1 in 4 teachers. And with one year of data, the error rate rose to 35 percent.
Harris points out that in Florida's system--which will label teachers as "highly effective," "effective," "in need of improvement" or "unsatisfactory"--minor differences of just one or two points could separate the teachers classified as effective from their colleagues deemed in need of improvement.
William Sanders, the statistician considered to be the "grandfather" of the value-added model in education, said this approach can target that extra professional boost to teachers who are rated poorly early in their careers, but who have the potential to improve. A study by SAS Research, the software firm where Sanders now works, found that half of the teachers who scored poorly after two years improved after five more years.
In terms of the actual formula, Sanders maintains that complexity is a good thing. He said any formula must include multiple years of student data to be reliable.
"A lot of people are arguing the methodology used must be so transparent that a teacher can go to their dining room table with a $2 calculator and scratch pad and calculate her value-added," Sanders said. "What I say to schools and districts is: To trade simplicity of calculation for reliability is a devil's bargain."
FORECASTING THE FUTURE
How will this new data-driven approach play out for teachers and their careers--and for students in the classrooms? Critics and proponents are sharply divided.
Officials with the state DOE say it will help teachers--and in turn, their students--improve.
"With value-added, we're trying to give better information to teachers about their student performance so they can improve their instruction," said Kathy Hebda, the state's deputy chancellor for educator quality.
Rep. Fresen envisions the new system giving incentives to teachers to move from high-performing schools to struggling ones where their students can make bigger gains and they can earn more money.
Critics contend that the opposite will happen: Teachers will not want to teach at low-performing schools or teach students who are not poised to score well on standardized exams because the stakes are too high or the system is unreliable.
"Many excellent teachers will get poor ratings, and many mediocre teachers (who are good at drilling) will get high scores," Diane Ravitch, a former U.S. Assistant Secretary of Education and fierce critic of tying teacher pay to student test scores, wrote in an email to The Miami Herald.
Ravitch said that placing so much emphasis on tests will lead to teaching to the test, gaming the system and cheating, and the new system will be a "massive waste of money."
Children will feel extra pressure, too, according to some experts and parents critical of standardized tests.
"Depending on the age and level of students, they may not realize that their test score has a connection to a teacher's salary, but they are going to feel the effects indirectly in their schooling," said Elisabeth Cramer, an education professor at Florida International University who has studied the effects of testing on students.
"Now that that there is going to be added emphasis from the teacher's perspective of performing well on these exams, students are going to feel that extra pressure."
This story also appeared in the Miami Herald on Nov. 6, 2011.
This story is part of a series by The Hechinger Report and newspapers in Florida. Stories have appeared in both the Miami Herald and Bradenton Herald. Read the entire package here.
You can also read our award-winning series from 2010 in partnership with the Milwaukee Journal Sentinel that took an in-depth look at teacher quality and effectiveness.