In an Ikea-sized warehouse turned de facto crime lab last fall, professor Gregory Cizek got his first look at the Atlanta test papers that would beget an education scandal of historic proportions.
Cizek, who teaches at the University of North Carolina at Chapel Hill, is a leading figure in psychometrics, the obscure field of mental measurement that includes setting and deciphering testing standards. He is often asked to seek out proof of tampering with student work. This case, however, was different.
In the Indianapolis warehouse, far from both his office and the schools where the suspect tests were taken, he saw clear evidence of what has become the most widespread episode of cheating ever documented in U.S. public schools, one which has diminished one of the nation's few education success stories of the past decade.
"Here you have a kid, this fourth-grader who sat down to take a test, who wrote his name on top of an answer booklet," Cizek recalls. "You see it was obviously changed through an awful lot of erasing. That's when you say, 'Something is going wrong here.'"
It was a long first day on the case for the professor, whose eyes burned as he left the warehouse on Sept. 20. Amid reams of stored paper and gigantic scanners, he had examined 1,000 answer sheets for the 2009 Criterion-Referenced Competency Test -- a standardized test administered in Georgia -- looking for signs of a teacher or principal erasing wrong answers, filling in right ones and trying to make them look like a student's work.
He found a lot of signs. Further erasure analysis, coupled with interviews of educators from flagged schools, led investigators to implicate some 178 educators in 44 of the 56 schools examined. The resulting report, released by Georgia Gov. Nathan Deal (R) last month, found "systemic misconduct within the district as far back as 2001" and concluded that "thousands of school children were harmed by widespread cheating in the Atlanta Public School System."
The 413-page report reads like a thriller, with tales of teachers holding "erasure parties" and principals publicly humiliating their employees. "A culture of fear and a conspiracy of silence infected the school system, and kept many teachers from teaching freely about misconduct," the report's authors concluded.
Unlike some of the other investigators taking a hard look at Atlanta teachers and principals, including former Georgia Attorney General Mike Bowers, Cizek isn't in law enforcement -- he's an educator himself. So finding evidence of cheating wasn't exactly a pleasure.
"The first time you see an answer sheet where, on a test with 40 questions, there was an average of 18 answers changed from wrong to right in a single classroom, it's not really a 'eureka' moment," he says. "It makes you feel awful.
The report has shaken the school system on both a local and national level because Atlanta's incredible test gains had garnered wide praise. Those scores' integrity had been bolstered by previous investigations spearheaded by the local business community without much incentive to dig deep.
On a national level, the cheating story cuts to the heart of a major education policy debate over accountability. The Atlanta report's conclusion that cheating resulted from a culture of fear, one spurred by rising test-score targets, fuels the argument that policies determined by test scores provide perverse incentives that are not in the best interests of students.
Those in favor of the rising-targets model argue that increases in test security are all that is needed. They assert the model has produced years of gains in the ongoing struggle to add discipline to the field of education and should not be changed because of one case they deem an outlier.
That debate has particular import this year. Roughly 30 states have begun or are about to shift their teacher evaluation systems to rely, by various degrees, on standardized test scores. (Upon receiving $400 million from the federal Race to the Top fund that is partly responsible for that shift, Georgia promised to evaluate educators under a system that counts scores for half of their reviews.) Congress has begun working to reauthorize No Child Left Behind, the sweeping education law notorious for first shifting national priorities to emphasize uniform high-stakes tests.
Atlanta's story has its echoes throughout the country. Washington, D.C., is currently under local and federal investigation for test tampering -- one teacher has already been fired. In the weeks following the Atlanta report's release, news outlets in Pennsylvania and New Jersey published government-issued erasure reports that raise similar red flags in those states.
Much more cheating appears to go unreported or unrevealed. States routinely conduct erasure analyses without releasing their results, according to Cizek. And Huffington Post reports on the Atlanta scandal have yielded several tips about suspicious testing practices in a number of states.
WHY TEACHERS CHEAT
While he has called for an overhaul of No Child Left Behind's narrowly-focused exams, U.S. Secretary of Education Arne Duncan's policy prescriptions have stressed judging teachers by student test scores, in addition to classroom observations and other such measures.
"To be sure, there are lessons to be learned from these jarring incidents, but the existence of cheating says nothing about the merits of testing," Duncan wrote in an op-ed for the Washington Post. "Instead, cheating reflects a willingness to lie at children's expense to avoid accountability -- an approach I reject entirely." Duncan recently gave a speech to teachers in which he called for a "national conversation" on merit pay, which would base teacher compensation on student performance.
But the Atlanta report's authors were less skeptical that tying the merit-pay system to annual standardized tests has skewed teacher incentives.
The report concluded that Atlanta Public Schools "put unreasonable pressure on teachers and principals to achieve targets. A culture of fear, intimidation and retaliation spread throughout the district [and] ... emphasized test results and public praise to the exclusion of integrity and ethics."
Those incentives dovetail with educators' natural desires to help their students, said Dan Ariely, a Duke University behavioral economist. While professionals faced with incentive systems generally try to balance the benefits of cheating with "wanting to look in the mirror and feeling good about themselves," Ariely said, "teachers have a biased view of the world. They want kids to be successful."
Ariely recently co-authored a National Academies paper that concluded the last decade of high-stakes testing policies led to very little learning.
"There's no excuse, but we're taking just one test a year," said Vicky Davis, a private school teacher in Georgia who runs the website Cool Cat Teacher Blog. "[Teachers'] funding and their lives and their money is based on that. When the stakes are extremely high and it's very competitive but you add in the fact that the teachers feel that they don't have control over the results, some will cheat.
Cizek, the testing forensics consultant, flatly rejects Ariely's contention that "cheating is very complex."
"When there's pressure to make deadlines, some people cut corners, like [plagiarizing New York Times reporter] Jayson Blair. People like [former baseball player] Roger Clemens take steroids. When there's political power, there's voter fraud," Cizek said. "Wherever there's incentives to be successful, people will find ways to cut those corners."
And Cizek conceded that the incentives to cheat have increased with a decade of education policy that uses scores on narrow standardized tests as a factor in many school managerial decisions, such as school funding, and in some cases, teacher pay.
"The stakes are higher than they have been in the past," he said. "K-12 has not had an accountability system until the last 10 to 15 years. That sort of newness has really caused a ripple in the system."
'IT WASN'T JUST FAMILIAR'
Cizek predicts that as education gets used to being measured, cheating will normalize and decline. Long before his trip to Indianapolis, he had seen cheating firsthand.
Cizek was teaching a course on testing and grading to master's students at the University of Toledo in 1997. He proctored his own final exam, with 60 multiple-choice questions followed by eight essays. Like any good proctor, he had students sit a seat apart with their books out of sight, and didn't notice anything suspicious during the test itself.
While he was grading, though, he thought he was having déjà vu. One essay felt familiar. "It wasn't just familiar -- I flipped back, and it was word-for-word identical to another," he says.
Then he came across another identical essay, and another, and another. The students with the same essays had each answered 59 of 60 multiple choice questions correctly, but missed the same one. Paper-watching wasn't possible, though, because they were sitting far away from each other during the exam. "They said it was a miracle," Cizek recalls.
The professor found a simpler answer. He examined the exams further and discovered extra staple holes in the suspect papers, which were attached to copies of the exam fuzzier than the ones he handed out in class. Their pages had identical smudges. And their essay pages were creased, lengthwise and across.
Cizek's conclusion, six months later: "They duplicated the entire test after breaking into my office. They pre-wrote their essays, folded them up, and stuck it in their pockets so that they could pull them out, remove the staple, slide the blank essay question out, slide their completed version in, and push the staple back."
After a three-year series of hearings, the cheaters ultimately failed the class. But the experience changed the course of Cizek's career.
He tried and failed to find a guide on detecting classroom cheating, so he wrote one himself. He took a sabbatical and traced academic cheating back to its origins. At Princeton University's East Asian Library, he found a garment that applicants would use to cheat the Chinese civil service exam -- with compositions based on Confucian essays written into its long sleeves.
After publishing his book, Cizek became a go-to consultant for cheating cases. He first encountered teacher cheating at a casual meeting in Michigan 12 years ago, where a school principal said, "Before we send our tests in, we check our tests for stray marks." Then he winked, and added, "like wrong answers." Cizek said he could tell from the principal's tone that he wasn't joking. "I didn't know what to think at the time," he said. "It was clear that they're not really looking for stray marks. They're looking to change wrong answers to correct ones."
Cizek's first consulting work on suspected teacher cheating came a few years later in New Jersey, in the pre-NCLB era before the dawn of state standardized tests. Schools used their own achievement and ability tests to track learning. In one classroom, the ability scores were much higher than the achievement scores. Cizek helped discover that one teacher changed all the students' answers, but had cheated on the wrong test. "She made them all look like underachieving geniuses," Cizek said. The district asked him to conduct statistical analysis, and the teacher was fired. She later found work teaching in another district.
Cizek cautions that his analysis alone is not definitive proof of cheating. "My explanation can't prove that the teacher did anything," he says. "But it can prove that it wasn't just random, or just a fluke, here or there. You form a hypothesis." In certain instances, superintendents have apologized to schools whose tests were prematurely invalidated by falsely alarming erasure analyses. In Atlanta, however, confessions by educators confirmed the cheating hypothesis.
RIPPLES ACROSS THE COUNTRY
Reports of educators cheating on behalf of students on standardized tests have spiked in recent years.
"I've never seen so many cheating scandals as there have been in the last few years," said Diane Ravitch, a New York University education historian and former U.S. Assistant Secretary of Education under Presidents George H.W. Bush and Bill Clinton.
Since her time in the executive branch, Ravitch has become the face of a loud group of critics of what she describes as the corporatization of education policy. "As we get closer to this deadline of [100 percent proficiency under NCLB by] 2014, it's not surprising that there are schools and districts where these things happen again and again," she said.
Earlier this year, a USA Today investigation found suspicious score improvements and erasure patterns on Washington, D.C., standardized tests. The reports led to heightened scrutiny of former district schools chancellor Michelle Rhee, a divisive figure in education policy who often boasts of score gains under her tenure. District authorities have begun a formal investigation, joined by federal agents following the release of the Atlanta report.
The Atlanta report came a month after a smaller episode in Baltimore, where Andres Alonso, CEO of Baltimore's schools, announced that his office had found evidence of cheating over the previous two years at two district elementary schools. Alonso then publicly promised that Baltimore's next standardized tests would have the most "extraordinarily transparent set of scores of any urban district in America."
The week after the Atlanta report's release, it was Pennsylvania's turn. The blog Philadelphia Public Schools Notebook published an erasure report the state conducted on its 2009 standardized test scores. That report flagged 60 schools statewide with suspicious gains and erasure marks on standardized exams. "The odds that the wrong-to-right erasure patterns that showed up on Roosevelt [Middle School]'s 7th grade reading response sheets occurred purely by chance were slightly less than 1 in 100 trillion," the blog reported.
Pennsylvania Department of Education spokesman Timothy Eller said his office only became aware of the report after the blog had requested it. "We're now at the first stage of an investigation process," he said. "We're having the districts named look into it and explain it to the department within 30 days. The reports that come back will be reviewed by the DOE." Eller added that Pennsylvania teachers fired for cheating lose their teaching certification.
The federal government has noticed the spate of cheating cases. "Folks are really paying attention to this," Duncan told The Huffington Post in light of the Atlanta and Pennsylvania revelations. "There's a greater awareness of the issues and trying to do things the right way. We put out guidance to states on this. You've got to take the state tests very seriously. You can't cheat children. You can't hurt children. That's exactly what you're doing."
Shortly after Alonso's announcement, Education Secretary Duncan addressed cheating in a letter to state superintendents of education:
I am writing to urge you to do everything you can to ensure the integrity of the data used to measure student achievement and ensure meaningful educational accountability in your State. As I'm sure you know, even the hint of testing irregularities and misconduct in the test administration process could call into question school reform efforts and undermine the State accountability systems that you have painstakingly built over the past decade.
Yet not long after the Pennsylvania report saw the light of day, the Asbury Park Press published the results of recent erasure analyses in New Jersey, which flagged irregular erasure marks on tests from several schools in the state.
"There's a tendency to look at this and say there must be something going on in these schools," said Justin Barra, a spokesman for the New Jersey Department of Education. He added that having students review their answers is good testing practice, and that the erasure report is only one investigative tool. "This is forensic evidence," he said. "We're trying to piece together a pattern."
New Jersey's education commissioner, Chris Cerf, sent a letter to districts notifying them of the erasure report and of his plan to investigate the schools with the most erasure marks. Traditional public schools with high erasure rates across the board -- in addition to three charter schools -- will be investigated by the state, and schools with high erasure rates in only one grade will be investigated by their districts.
Last week, the Connecticut Post wrote of a new cheating case in Waterbury, Conn. There, 17 educators at Hopeville Elementary School are currently on administrative leave while state officials investigate alleged cheating on state tests in spring 2010. The expense of the investigation there has led the state's education commissioner to call for legislation that "enhances the liability of professionals who engage in this kind of work."
THE TIP OF THE ICEBERG
Duncan and others have repeatedly stressed that the majority of teachers do not cheat. That appears to have been true in Atlanta. Even so, the lack of incentives for schools to report cheating strongly suggests that more such incidents remain hidden from the public. It is unclear, however, whether the Atlanta scandal will encourage others to speak up or whether its wide-ranging fallout will have a chilling effect on potential whistleblowers.
"On the one hand, we've got new administrations at the state level which are quite willing to reveal problems with the previous administration," said Jeffrey Henig, a professor at Columbia University's Teachers College. "By the same token, in a lot of states there's going to be a preference to not find out if there's large-scale cheating, to avoid undertaking this kind of investigation, as the result will be questioning their own claims of academic success." In other words, politicians respond to incentives, too.
Getting something for nothing is a powerful incentive. According to Cizek, most states' testing companies regularly perform erasure analyses that simply are not widely disseminated. "It's a no-cost option," he said. "It's a flip of a switch."
The results, Cizek said, are usually gathered as a separate report that's stored or filed and sent to the state. There is usually little follow-up unless something like a cheating complaint triggers an investigation.
Few erasure reports see the light of day. The Atlanta Journal Constitution spent years questioning suspicious score gains and filing public records requests before the scores became a national scandal.
To unearth Pennsylvania's report, the Philadelphia Schools Notebook had to request a data package on testing, and received in May a convoluted file that Eller, the Pennsylvania DOE spokesman, said his office had considered useless. Within the file, however, the Notebook found a 44-page erasure analysis, which the blog passed onto Eller. "I provided that directly to the secretary of education," Eller said.
Pennsylvania first ordered an erasure report in 2009, but not 2010. "It's my understanding that it was suggested or offered by our contractor that they could do this analysis," Eller said. Since the revelation of the 2009 report, Gov. Tom Corbett (R) ordered an analysis of last year's results as well.
In New Jersey, the Asbury Park Press filed an open records request for a report, but had to sue the state to obtain a copy without redacted school names. According to Barra, the state DOE spokesman, his department was hesitant to release names because the results are inconclusive. "There's a danger in releasing the report: You start to tag schools that may not actually have problems," he said.
Yet Barra added: "I'm very happy this is something we can talk about publicly. We need to talk about the integrity of the data across the country and this is a vehicle to do that."
Huffington Post reports on these cases have prompted observers to write in about their own experiences with suspected educator cheating. One reader from California said the gains in his school outpaced apparent student learning, and testing security seemed lax. A resident of Bucks County, Pa., reported that during tests, teachers check papers and demand that students change their answers. And a parent in Louisville said her daughter had complained about her kindergarten teacher changing her answers -- and, upon review of the papers in question, found the teacher's handwriting scrawled over her daughter's own.
But such concerned citizens see few incentives to air their grievances publicly. In 2005, local members of the American Federation of Teachers tipped off Georgia education officials about the cheating in Atlanta. They were reprimanded.
AFT President Randi Weingarten cites that case as having a chilling effect on further investigations. "When a whistleblower says something, they know that they're likely to be retaliated against," she told HuffPost. "I'm glad that the Atlanta situation -- even though we don't know all the facts -- that it's getting so much attention, because juxtapose that to what's going on in Washington D.C., which nobody wants to investigate."
She said that the high stakes put on teachers coupled with environments generally hostile to whistleblowers puts them in a "catch-22 situation where there's no due process."
"We need to have multiple measures," Weingarten said. "We need to have measures that are aligned to what kids need to be able to do in the 21st century."
This can change: No Child Left Behind is up for reauthorization in Congress. While progress on that front has stalled, the law will eventually either be remade or set aside. Various efforts, such as the crafting of a uniform curriculum known as the Common Core, seek to broaden the narrow exams that have resulted in a laser-like focus on reading and math. But the current No Child left behind negotiations seem to align with the federal government's view that high-stakes tests do not cause or encourage cheating. And while there is constant talk of the creation of a "next generation" of exams, there has been less about federal whistleblower protections, a signal that curtailing cheating is not a major legislative concern.
CHEATING BEGETS CHEATING
In Atlanta, now that the veneer of dramatic success is gone, the school system has had to face criticism and calls for soul-searching. The schools have a new leader in interim superintendent Erroll Davis, who has reshuffled central offices. School officials have held numerous town halls to answer to public outrage, told teachers named in the report that they would have to resign or be fired, and hired an outside testing company to administer future standardized tests.
Criminal cases have been opened. The state is about to expand its investigation into Dougherty County. Duncan said his office is "looking at" Atlanta Public Schools, and the Georgia Department of Education said it might revoke federal funding allocated by test scores to the schools accused of fudging results. A state politician called for legislation that would require implicated teachers to return any bonus money earned based on test results.
Atlanta schools are back in session Monday, with the scandal still on many minds. More important for the students, the Atlanta public school system is up for reaccreditation this fall, and the cheating report has the district worried. Since most colleges don't accept students from unaccredited schools, the cheating by a few teachers in a few schools could jeopardize the college applications of students districtwide.
"The whole district is getting painted with this broad brush, when there really are some remarkable teachers here," said Atlanta attorney Melinda Moseley, whose daughter Samantha, 14, did not attend one of the schools flagged in the report. "These bad apples, the ones who cheated, they could prevent my little girl from getting into school."
Atlanta's accreditation could yet survive. But the effects of those high-stakes tests are likely to linger.
"Cheating begets cheating," said Ariely, the Duke behavioral economist. "Once you cheat in some way, the next act of cheating is easier. We're not dealing with separate instances of cheating but we're dealing with things that have accumulated."