10/19/2010 09:14 pm ET | Updated May 25, 2011

The Department of Faith-Based Education Initiatives

Does that title surprise you? It shouldn't. When a department and its leadership are advocating for courses of actions either lacking research supporting effectiveness or -- even worse -- where the research clearly demonstrates that the course of action does not work or provides inaccurate and unreliable information, to insist on pursuing such a course can only be because one has a blind faith, is operating on the basis of theological or ideological purity.

That is not scientific. It certainly should not be how the United States Department of Education operates.

Sadly, that is the clear pattern of this administration under the leadership of Arne Duncan.

Consider Race to the Top (RttT). States that wish to receive that infusion of funds were faced with three critical issues. They had to significantly tie teacher evaluation to student test scores. They had to lift the caps on the number of charter schools. Finally, states had to turn to one of four methods of addressing "failing" schools.

Let's take the "failing" schools first. Those determinations are being made on the basis of student scores on tests that by the administration's own admission are seriously lacking -- after all, why else commit more than $300 million to two consortia of states to devise new and better assessments? Yet in the meantime both under RttT and the Blue Print states are supposed to decide what the worst five percent of schools are using the current tests. The methods of restructuring from which states can choose are described in this speech by Secretary Duncan. There are several problems. First, the claims he makes for success in Chicago are not accurate -- as a number of sources have noted, the higher scores for the schools he cites were not with the same students tested for the lower scores. Second, as Diane Ravitch and others have noted, there is no evidence that following any of these methods has ever improved the performance of schools.

For some reason this administration is willing to insist on expansion of charter schools without sufficient controls. Yes, there are some charter schools that do very well. The most thorough examination of charter schools nationally is the CREDO study out of Stanford University. In that study we learn that 17 percent of charters perform better than the public schools from which they draw, 37 percent perform worse, and the remaining 46 percent perform no differently. The requirement to lift the caps on charters has no requirement for control for quality. Based on the record, there is better than a 2-1 chance that the additional charters will perform worse than there is that they will perform better than the local public schools.

Finally there is the insistence upon tying teacher evaluation to student scores on tests, despite the wealth of evidence that we lack a reliable way of ascertaining the teacher effects, even with value-added methodologies. The Department of Education should know this, having commissioned a study from Mathematica which found that attempting to determine a superior or inferior teacher using value-added methodologies resulted in a 37 percent mis-classification when using two years of data, 26 percent when using three years, and even with 10 years still had a 12 percent mis-classification. As the authors of the report noted;

These results strongly support the notion that policymakers must carefully consider system error rates in designing and implementing teacher performance measurement systems based on value-added models, especially when using these estimates to make high-stakes decisions regarding teachers (such as tenure and firing decisions).

A subsequent study just released offered clear reasons -- teacher value-added scores, regardless of method used, were unstable, and largely an artifact of the students for whom the teacher was responsible: in one case a teacher was classified in the lowest decile (10th) the first year and the highest the second. Why? The first year the teacher had a very high rate of English Language Learners, and the second year did not. As the notable authors of this study wrote,

we conclude that caution should be exercised in using student achievement gains and value-added methods to assess teachers' effectiveness, especially when the stakes are high.

Expand charters without controls for quality even though overall they do not perform better than regular public schools. Reconstitute/reorganize schools using methods that are not shown to improve education or teaching. Evaluate teacher performance using measures that the experts demonstrate are neither stable or reliable. And argue to the American people that the Department of Education is thereby improving American schools and the learning of our students? Where's the evidence?

And absent the evidence, on what basis other than ideology can one justify demanding such changes?

So perhaps my title is appropriate: Arne Duncan should have his job retitled as the "Secretary of Faith-Based Education Initiatives."