by Joshua Bleiberg and Darrell West
The integrity of student assessments has taken major hits in the past year. Former Atlanta School District Superintendent Beverly Hall faces a bevy of criminal charges along with other educators related to falsifying test scores. In Washington, D.C., former Chancellor Michelle Rhee has faced accusations of cheating in her district. A memo sent to district officials described unusually high erasure rates, which often indicates cheating. In Philadelphia, two administrators reportedly admitted to investigators they corrected wrong answers. A USA Today investigation of test scores across the country found 1,610 schools where year-to-year changes in test scores were more than three standard deviations larger than the average statewide gain.
The causes and antidotes for cheating are a subject of intense debate. Some believe there is a direct relationship between the rise of standardized testing and cheating. Accountability policies have pressured educators to raise test scores. The pressure creates incentives for teachers to cheat, according to critics of high-stakes testing.
Accountability advocates, however, argue that tests provide invaluable information. Many recognize greater reliance on quantitative metrics will subject the measure to corruption and distort the processes it is intended to measure. They believe better security for test materials and test proctors will expose cheaters. They also contend that tests which better reflect what students do or do not learn will lessen instances of cheating.
This debate misses the point as standardized testing is a double-edged sword. Testing provides a wealth of data on student performance. However, there is mounting evidence that accountability pressures have incentivized cheating and reliance on standardized tests has lessened their efficacy.
Innovative testing technologies like Computerized Adaptive Testing (CAT) represent a way to improve the accuracy of assessment and address cheating concerns. A core advantage of CAT is its potential to refocus learning on instruction. The adaptive nature of CAT can help minimize Campbell's corruptive pressures and reduce teaching to the test. CAT uses an algorithm to choose test items based on the students strengths and weaknesses. Every student takes a different test when using CAT. This covers a larger domain of knowledge and decreases the number of items tests have in common. Currently educators can look at past fixed form tests to predict which questions are likely to reappear. Teachers may then narrow their teaching to those subjects. Adaptive testing disrupts teaching to the test because tests have fewer items in common.
CAT makes cheating more difficult without costly security measures. Students take the test online making it impossible for teachers to erase incorrect answers. CAT also eliminates inappropriately administered testing accommodations. CAT can provide testing accommodations like text to speech, speech to text, and text magnification. Currently educators provide these services, which creates an avenue for cheating. CAT can also limit cheating from students sharing items. CAT tests have fewer items in common the fixed form tests making it difficult to share questions. The main virtue of CAT is the removal of cheating opportunities without costly security measures or unnecessary surveillance of teachers.
CAT has numerous advantages over paper based tests. CAT tests scores have greater reliability for very weak students and very strong students. Built in testing accommodations increase scoring accuracy for students with disabilities. CAT also costs less to administer and takes less time for students to complete. Adoption of CAT is critical to improving the quality of assessments
The success of CAT will depend on the Common Core State Standards (CCSS). Two different consortia the Partnership for Assessment of Readiness for College (PARCC) and the Smarter Balanced Assessment Consortium (SBAC) are in the process of developing the CCSS assessments. PARCC has committed to using fixed form computer based tests while SBAC will use CAT. SBAC has 25 member states and represents about 4 out of 10 American students. Alabama, Minnesota, Nebraska, Texas, and Virginia do not belong to a consortium. Several consortium members have considered not adopting the new assessments. PARC has left the option open to add an adaptive supplement to their test.
Too often, debates in education policy devolve to the point where advocates argue past each other. Experts will debate the merits of accountability policies for years to come. Assessment policy involves a series of tradeoffs including the relationship between teacher evaluations and cheating. The debate, though, should not derail improvements to existing systems. CAT is secure, cheap, and accurate, and every student and teacher deserves those benefits.
Joshua Bleiberg is the center coordinator for the Center for Technology Innovation at Brookings. Darrell West is vice president of Governance Studies and the founding director for the Center for Technology Innovation at Brookings. West is also the author of Digital Schools: How Technology Can Transform Education.