iPhone app iPad app Android phone app Android tablet app More

Featuring fresh takes and real-time analysis from HuffPost's signature lineup of contributors
Michael J. Petrilli

GET UPDATES FROM Michael J. Petrilli
 

The "Teacher Effectiveness Gap" Was Just a Myth: 3 Implications

Posted: 02/28/2012 11:36 am

UPDATE: This fantastic Gotham Schools article explains that New York's rating system was designed to guarantee that "effective" and "ineffective" teachers would be found all over the city. Which renders the New York Times story--and my post--basically moot.
Still, this wasn't the first bit of evidence showing that we might not have a teacher effectiveness gap, or at least much of one. This rigorous CALDER study, in particular, found that:

The average effectiveness of teachers in high-poverty schools is in general less than teachers in other schools, but only slightly, and not in all comparisons. The authors also find differences in within-school-type variation in teacher effectiveness in nearly every comparison. These differences are largely driven by the longer tail at the bottom of the teacher effectiveness distribution in high-poverty schools. Teachers at the top of the effectiveness distribution are very similar across school settings.

So the evidence on the lack of a gap isn't as open and shut as my post implies. But it certainly appears likely that the gap is much smaller than we once thought -- which does call for pushing the pause button on massive efforts to move teachers around.

______

The finding -- reported by the Times this weekend -- that really good, and really bad, teachers are evenly distributed around New York City is jaw-dropping news. It upends everything we thought we knew about teacher quality, especially the notion that our achievement gap is caused in large part by a "teacher quality gap," with the worst teachers clustered in the neediest schools. But they aren't. So now what?

Let me stipulate that this finding might be incorrect (though previous analyses have come to similar conclusions). Maybe it's harder for teachers in affluent schools to show strong value-added gains, because their students are already topping out on the tests. Perhaps student mobility is making teachers in high-poverty schools look better than they really are. (Their worst students don't show up for testing -- or have already moved onto another school.)

But assume it's true. What are the implications?

  1. Affluent schools are spending more for their teachers -- but they aren't getting better results. We know from research by Marguerite Roza and others that low-poverty schools tend to employ older, and thus more expensive, teachers than their poorer counterparts. We all know the system features that enable this to happen -- seniority bumping rights, a single salary schedule, etc. But these older, more expensive teachers aren't getting stronger value-added gains than their younger, less expensive peers. This is more evidence that the teacher qualifications we can measure (and for which our salary schedule pays extra) -- degrees, years of experience, etc. -- are not related to effectiveness.
  2. Affluent schools might be getting less value added by choice. It's perfectly reasonable for educators and parents in affluent, high-achieving schools to trade-off sky-high math and reading scores (and/or test score gains) for other values, like more time for art, music, science, history, and P.E.
  3. A focus on "redistributing effective teachers" from affluent to poor schools seems misguided, or worse. It turns out that effective (and ineffective) teachers are everywhere. Which means that we should push the pause button on efforts to move teachers from one kind of school to another -- efforts that many reform groups want embedded in the next Elementary and Secondary Education Act.

We have a lot of problems in K-12 education to address. Let's be grateful that we can take "closing the teacher effectiveness gap" off our to-do list.

Originally published on the Fordham Institute's Flypaper blog.

 

Follow Michael J. Petrilli on Twitter: www.twitter.com/MichaelPetrilli

UPDATE: This fantastic Gotham Schools article explains that New York's rating system was designed to guarantee that "effective" and "ineffective" teachers would be found all over the city. Which rende...
UPDATE: This fantastic Gotham Schools article explains that New York's rating system was designed to guarantee that "effective" and "ineffective" teachers would be found all over the city. Which rende...
 
 
  • Comments
  • 26
  • Pending Comments
  • 0
  • View FAQ
Comments are closed for this entry
View All
Favorites
Recency  | 
Popularity
07:47 PM on 03/01/2012
The 800 lb. gorilla in the room is student behavior. I think that students come into school in the primary grades unprepared to learn for many different reasons. It would be better to "graduate" students who are ready to work in an academic environment instead of changing the academics and lower standards. Some students "grow out of it" while others never do because of limited guidance at home. Why should all the students who want to learn suffer because of a few bad kids. This way the playing field would be level and we could evaluate teachers. After all, you can't pick your students or your relatives.
HUFFPOST SUPER USER
ErikKengaard
10:55 AM on 02/29/2012
Perhaps the students - or their parents - are the problem. What then?
photo
nypoet22
Psychology Ph.D., Civics Teacher, Songwriter
11:41 PM on 02/29/2012
community support is a proven intervention.
06:54 PM on 02/28/2012
I think the value added scores are nearly meaningless to begin with. But, even if your are a believer, the reasoning in this article is faulty.

High poverty students with a lot of mobility to not benefit teachers in those areas. Where do you think they move to? Affluent suburbs. If you & I teach in high poverty schools, some of your students move to my school, some of my students move to your school, and we both lose in the deal due to the lack of continuity for the students moving.

The value added formula is very complicated and gives extra weight for increasing the performance of high poverty students among other things. The reason teachers in all these different settings come out about the same, on average, is because the value added formula is calibrated to make this happen. This is like me using a strict curve in my class and saying my test was about right because I had just as many students that did really well (A) as failed. It has nothing to do with the test or the students knowledge, the grading system I used ensured the same number of A's and F's (not that I actually use a strict curve).
photo
HUFFPOST SUPER USER
tultican
Thomas Ultican, MEd. BS Mecahnical Engineering
07:32 PM on 02/28/2012
Value added is unproven, but the data that exists indicates it is unstable and rates teachers excellent one year and poor the next. An unbiased researcher would not make any conclusions based on value added data. The Fordham Institute has a definite bias!
photo
nypoet22
Psychology Ph.D., Civics Teacher, Songwriter
11:41 PM on 02/29/2012
the reliability of VAM is .35 - reliability is equivalent to a pearson correlation, the square root of the variance accounted for, i.e. .35 x .35 = .1225 (12.25%)

12.25%(!!!)
06:38 PM on 02/28/2012
Where to begin.

"Maybe student mobility is making teachers in high poverty areas look better (they move to other schools)". Where on earth do you think these high mobility students in high poverty areas move to? Affluent schools? They are shuffled between different teachers in different high poverty areas. If you & I are teachers in high poverty areas, some of my students will move to your school, and some of your students will move to my school. Neither one of us is helped by this.

Teachers in high poverty areas get, on average, about the same ratings as teachers in other areas because the very complicated value added formula was essentially set up to make this happen. The value added scores factor in the characteristics of the students. Moving a high poverty student up 5 percentile points gets you a higher rating than moving a non-impoverished student up 5 percentile points because it is more difficult to accomplish, on average. How do you decide how much of a "curve" to give the teachers of high poverty students? Well you give enough of a curve so that they come out about the same as everyone else. Maybe the high poverty area teachers are better, maybe they are worse, maybe the are the same - there is no way of knowing from value added scores.
This user has chosen to opt out of the Badges program
photo
03:09 PM on 02/28/2012
What a sham! Shame on you- someone who's not even a teacher writing this article... Typical. Who the hell is this guy even?

How exactly does test scores even evaluate a teacher? Is this truly the ONLY way we can evaluate them?

Does this take kids backgrounds at all into consideration? The fact that half our nations' kids fail to even motivate themselves to graduate? That most families now are single parent? Or are too poor to enrich the kids? Or that they are performing years below grade level when they enter classrooms?

AMERICA NEEDS TO GET OUT OF ITS DENIAL. Schools are microcosms of their communities. They are not only places of transformation, but are reflections of the priorities of those communites. I am an experienced teacher of 15 years in inner city schools and I have seen the villification process go from bad to worse. I graduated the top of my class at NYU and yet every day, I get reminded of my failures. Thank you America. I'll be quitting soon. Enjoy the nightmare you made.
07:49 PM on 03/01/2012
Amen I say! I am a teacher who has a "knack" with behavior issues that other teachers won't even deal with. What are the measures for working with those students? No one wants them now, with this type of mentality we may as well just lock them up now.
photo
HUFFPOST SUPER USER
10YearTeacher
02:34 PM on 02/28/2012
So, this guy manages to find a way to still say we are paying teachers too much.
And how exactly is more time for art, music, and physical education NOT part of "adding value."
This guy's bias is so apparent even a 3rd grader can see it.
photo
nypoet22
Psychology Ph.D., Civics Teacher, Songwriter
02:33 PM on 02/28/2012
1. the conclusion that experience is unrelated to effectiveness rests upon the faulty assumption that "value-added" results are a valid or accurate measure of teacher contributions to student learning. they are neither. teachers account for (at most) 12.25% of the variance on standardized tests, and tests in turn represent an extremely narrow sample of what students learn.

2. yes, better schools whose students go on to future success don't care much about "value-added." this should be evidence enough that it doesn't add much actual value.

3. teachers are not a commodity that can be easily shifted from one school to another without disturbing the ecology of both. the quality of education in a classroom depends on many different variables, not the least of which is stability. creating instability in a school environment is generally harmful.
photo
HUFFPOST SUPER USER
tultican
Thomas Ultican, MEd. BS Mecahnical Engineering
07:42 PM on 02/28/2012
Of course experience in a classroom is crucial. If a conservative institute uses unproven statistical methods to reach conclusions any claim the institute wants to make can be made. However, if the claims were peer reviewed they would be thoroughly debunked.
10:33 PM on 02/29/2012
Last time I saw in-school impact on student standardized test score variance quantified (and admittedly, that's not quite the same thing as teacher impact), it ranged as high as 18%... or as low as 4%.

Which is just a quibble, as whichever number we choose, the fact remains that teachers have a small impact on that number and it's completely inaccurate to base their evaluations on it.
12:14 PM on 02/28/2012
Culture and Achievement Gap is caused by differences in three forms of culture.

1. Neighbourhood Culture
2. Family Culture
3. Peer Group Culture

Above cultures generate emotional states leading to non-compliance. These cultural and emotional factors determine achievement gap.

Non-compliant kids, (almost always from chaotic environments) are unteachable. Their behaviour turns teachers into behavioral managers. They destroy the educational opportunities of students they share classes with.

The needs of the non-compliant cannot be met in regular schools. They need alternative provision.

Avoidance of this reality is the reason that the schools fail and no solution is ever found. The schools cannot provide a solution. They are the problem.

The non-compliant require an unstructured, democratic, non-institutional environment. Not school.

Those who refuse to recognize this, help make things worse and worse. And that is what is happening. Things are getting worse and worse. Numbers of non-compliant grows and grows.
photo
jshop
Come together right now over them.
01:02 PM on 02/28/2012
Interesting analysis. I've never heard it explained like this, but it makes sense. Teachers (and schools in general) with a lot of misbehaving students spend more time trying to manage misbehavior and minimize disruption than teachers with fewer/milder behavioral issues to deal with. But then, this always was common knowledge to parents and educators alike; so, how and why did the notion that American public education is failing because of "bad" teachers and their unions get to be accepted as obvious truth...?
01:29 PM on 02/28/2012
'' Blame the liberals, they're all commies anyway, and atheists.''

It beats thinking!
06:32 PM on 02/28/2012
To answer more seriously. It is partly the rightist prejudice I was flippant about in my first reply but there is another reason.

Progressives are committed to an analysis of the educational crisis based upon social deprivation. If people are deprived they are vulnerable and therefore not to blame. And, if you dare to assert that the non-compliant are responsible, your narrative will be perverted by conservatives into one of the lazy and disaffected poor.

So, the truth cannot be faced. Because the truth is - the poor, the vulgar, the uneducated, the feckless, the disaffected and the chaotic are sending disturbed, alienated children into the system where they cause havoc.

Formerly, they could be intimidated into dumb obedience but with the move from traditionalist to humanistic pedagogy, rule-based systems of compliance came to be.
And, as we all now know (and many rationalize), the disaffected care not a whit for rules.

Why so sensitive to the compliance system? Because they are already savagely oppressed, the extra burden that the school imposes is intolerable. These non-compliant children are the real progressives. They demand democratization.

And progressive educators are now the new traditionalists.
Hard to bear: easy to deny.
photo
HUFFPOST SUPER USER
tultican
Thomas Ultican, MEd. BS Mecahnical Engineering
07:54 PM on 02/28/2012
I have worked in tough neighborhoods for a decade and I agree that changing the culture is the only real fix for the local community. On the other hand, I find that most students want to do well. However, standards based education precludes teachers from creating lessons based on the needs of their students. Instead the student, their school and their teachers are labeled as failures if that 9th grader cannot master algebra. The students are forced into classes that they are not ready to take and they give up. A student with no hope is a behavior problem. Trying to run classrooms from DC or Sacramento is a plan for manifesting bad pedagogy and alienated children.