The process of moving an educational innovation from a good idea to widespread effective implementation is far from straightforward, and no one has a magic formula for doing it. The William T. Grant and Spencer Foundations, with help from the Forum for Youth Investment, have created a community composed of grantees in the federal Investing in Innovation (i3) program to share ideas and best practices. Our Success for All program participates in this community. In this space, I, in partnership with the Forum for Youth Investment, highlight observations from the experiences of i3 grantees other than our own, in an attempt to share the thinking and experience of colleagues out on the front lines of evidence-based reform. This blog post is from Dr. Donald J. Peurach, Assistant Professor of Educational Studies in the University of Michigan's School of Education. Since 2012, Dr. Peurach has served as an advisor and contributor to the i3 Learning Community. As a researcher who focuses on large-scale educational reform, Dr. Peurach provides his perspective from the front lines.
As a participant-observer in the i3 Learning Community, I have had a front row seat on ambitious efforts by the U.S. Department of Education's Office of Innovation and Improvement (OII) to revolutionize educational innovation and reform. Others will soon have a glimpse, too, and the fate of the revolution may well rest on how they interpret what they see.
With its Investing in Innovation (i3) program, OII is investing nearly a billion dollars in the development, validation, and scale up of over one hundred diverse reform initiatives, all subject to rigorous, independent evaluations. In coordination with the U.S. Department of Education's Institute of Education Sciences (IES), results will be reported in the What Works Clearinghouse so that decision makers have high-quality information on which to base school improvement efforts.
For most people, their best glimpse of the i3-funded initiatives will come via these evaluation results. Preliminary reports from two scale-up recipients are largely positive: Reading Recovery and Success for All. This is not surprising. Both are well-established enterprises that have been refined through more than two decades of use in thousands of schools.
Additional evaluation results are soon to follow, from a broad array of initiatives not nearly as well established. History predicts that many of these results will be characterized by variability in implementation and outcomes that cloud efforts to determine what works (and what doesn't). But this, too, would not be surprising. Both researchers and reformers (including contributors to this blog) have long reported that efforts to establish and evaluate ambitious improvement initiatives have been challenged by interactions among the complex problems to be solved in schools, the uncertain research base on which to draw, and the turbulent environments of U.S. public education.
If historical precedents hold, the effect could be to leave OII's efforts politically vulnerable, as promises of revolution and equivocal results are not a good mix. For example, barely five years after finding support in federal policy, the comprehensive school reform movement met a quick and quiet death, as lofty promises of "break-the-mold" school improvement collided with equivocal evaluation results to contribute to a rapid erosion of political support. This was the case despite a small number of positive outliers having met high standards for evidence of effectiveness (including Success for All).
Yet new developments provide reasons for hope. Within the i3 Learning Community, reformers are collaborating to develop and manage their enterprises as learning systems that improve and persist in the face of complexity, uncertainty, and turbulence. Doing so includes critically analyzing implementation and outcomes in order to understand, explain, and respond to both successes and struggles. Similar work is underway in the Hewlett Foundation's "Deeper Learning" initiative.
Moreover, rather than passing summary judgment based on quick glimpses, researchers and policymakers are increasingly recognizing the struggles of reformers as legitimate, and they are interpreting equivocality in evaluation results as a reason to push still-deeper into the challenging work of educational innovation and reform. For example, some researchers are working hard to systematically study variation in program effects to determine what works, where, for whom, and why. With new support from IES, other researchers working inside and outside of the academy are advancing improvement-focused evaluation strategies that have great potential to reduce that variation.
Such efforts mark a great advance beyond a narrow focus on determining what works (and what doesn't). To be clear: Making that determination is, at some point, absolutely essential. After all, the life chances of many, many students hang in the balance. The advance lies in acknowledging that the road to positive results is far rockier than most realize, and that paving it smooth requires supporting reformers in learning to manage the complexity, uncertainty, and turbulence that have long been their undoing.
Indeed, from my front row seat, the revolution in educational innovation and reform looks to be just beginning, with increasing potential to coordinate new, improvement-focused evaluation strategies with more sophisticated impact evaluation strategies in both supporting and assessing educational innovation. Whether that is, in fact, the case will depend in no small part on what others make of the glimpses provided by forthcoming i3 evaluation results: what they make of outlying successes and failures, certainly; but, more importantly, what they make of (and decide to do about) the great, grey space in the middle.