It can't be repeated often enough: Standardized tests are very poor measures of the intellectual capabilities that matter most, and that's true because of how they're designed, not just because of how they're used. Like other writers, I've relied on arguments and research to make this point. But sometimes a telling example can be more effective. So here's an item that appeared on the state high school math exam in Massachusetts:
n 1 2 3 4 5 6tn 3 5 __ __ __ __
The first two terms of a sequence, t1 and t2, are shown above as 3 and 5. Using the rule: tn = (tn-1) plus (tn-2), where n is greater than or equal to 3, complete the table.
But perhaps you figured out that the test designers are just asking you to add 3 and 5 to get 8, then add 5 and 8 to get 13, then add 8 to 13 to get 21, and so on. If so, congratulations. But what is the question really testing? A pair of math educators, Al Cuoco and Faye Ruopp, pointed out how much less is going on here than meets the eye:
The problem simply requires the ability to follow a rule; there is no mathematics in it at all. And many 10th-grade students will get it wrong, not because they lack the mathematical thinking necessary to fill in the table, but simply because they haven't had experience with the notation. Next year, however, teachers will prep students on how to use formulas like tn = tn-1 + tn-2, more students will get it right, and state education officials will tell us that we are increasing mathematical literacy.[1]
In contrast to most criticisms of standardized testing, which look at tests in the aggregate and their effects on entire populations, this is a bottom-up critique. Its impact is to challenge not only the view that such tests provide "objective" data about learning but to jolt us into realizing that high scores are not necessarily good news and low scores are not necessarily bad news.
If the questions on a test measure little more than the ability to apply an algorithm mindlessly, then you can't use the results of that test to make pronouncements about this kid's (or this school's, or this state's, or this country's) proficiency at mathematical thinking. Similarly, if the questions on a science or social studies test mostly gauge the number of dates or definitions that have been committed to memory -- and, perhaps, a generic skill at taking tests -- it would be foolish to draw conclusions about students' understanding of those fields.
A parallel bottom-up critique emerges from interviewing children about why they picked the answers they did on multiple-choice exams -- answers for which they received no credit -- and discovering that some of their reasons are actually quite sophisticated, which of course one would never know just by counting the number of their "correct" answers.[2]
No newspaper, no politician, no parent or school administrator should ever assume that a test score is a valid and meaningful indicator without looking carefully at the questions on that test to ascertain that they're designed to measure something of importance and do so effectively. Moreover, as Cuoco and Ruopp remind us, rising scores over time are often nothing to cheer about because the kind of instruction intended to prepare kids for the test -- even when it does so successfully -- may be instruction that's not particularly valuable. Indeed, teaching designed to raise test scores typically reduces the time available for real learning. And it's naïve to tell teachers they should "just teach well and let the tests take care of themselves." Indeed, if the questions on the tests are sufficiently stupid, bad teaching may produce better scores than good teaching.
_____________________________________________
1. Cuoco and Ruopp, "Math Exam Rationale Doesn't Add Up," Boston Globe, May 24, 1998, p. D3.
2. For examples (and analysis) of this kind of discrepancy, see Banesh Hoffmann, The Tyranny of Testing (New York: Crowell-Collier, 1962); Deborah Meier, "Why Reading Tests Don't Test Reading," Dissent, Fall 1981: 457-66; Walt Haney and Laurie Scott, "Talking with Children About Tests: An Exploratory Study of Test Item Ambiguity," in Roy O. Freedle and Richard P. Duran, eds., Cognitive and Linguistic Analyses of Test Performance (Norwood, NJ: Ablex, 1987); and Clifford Hill and Eric Larsen, Children and Reading Tests (Stamford, CT: Ablex, 2000).
Follow Alfie Kohn on Twitter: www.twitter.com/@alfiekohn
Quite the contrary.
Standardized testing rewards drone like regurgitation of utterly meaningless facts - this is the perfect way to create the permanent underclass that the GOP want to fill their factories and businesses for third world wages. Young people educated in this way will have no life opportunities available to them. You can be sure the children of the rich are not educated this way.
Re #1: Seems like you can't distinguish between ST and how they're used.
Re #2: Many teachers are strikingly ignorant of content knowledge, and even worse when it comes to anything outside teaching.
Re #3: Seems like you can't distinguish between ST and how they're used.
Re #4: Seems like you can't distinguish between ST and how they're used.
Re #5: I'd rather see my kid do "superficial" test prep as opposed to superficial arts-and-crafts project
Re #10: Could be the other way around too.
As an example of a standardized test that encourages bad teaching, look at the SAT 2 for Physics. The test assumes a march through far too many topics. Such a "coverage" focused approach to high school physics has been shown repeatedly to result in little understanding of physics. Approaches to physics based on doing experiments and developing reasoning ability take much more time, but are shown to truly change the way students understand and view the world. Modeling Instruction (modeling.asu.edu) has a track record of producing some of the best results in the world, but it does not prepare students for the SAT 2. The upshot (in high school physics, anyway) is a stark choice between preparing for the test or actually learning physics.
So some tests have been developed. Who paid for them? We as citizens did (they were all created with NSF or local money invested in higher education). I guess we could call on Bill Gates to fund this kind of work, but he seems to prefer that I make one-take, error-filled youtube videos for free in my spare time. I do plenty in my spare time for free, but it won't be duplicating a poor teaching method on video.
Also, in isolation that question may appear quite abstract and irrelevant. However, the topic involved is quite useful. Recursive formulas similar to the one in the test question can be quite helpful in building even moderately complicated spreadsheets.
Finally you made an obvious typo in the example you provided (the parenthesis are misplaced). The typo completely changes the question - it becomes nonsense with your typo. It does not seem like you understand the notation involved in this question.
By the same token Alfie should be criticizing you if you read a book or even if you are able to. "Real" scholarship is to be found in writing books, and not reading them. I suggest that no one ever buy another one of Alfie's books.
>>they haven't had experience with the notation.
That's like saying that Billy is fluent in Spanish, he just hasn't had experience with the vocabulary.
Notation is a big part of mathematics.
>>Next year, however, teachers will prep students on
>>how to use formulas like tn = tn-1 + tn-2, more
>>students will get it right, and state education
>>officials will tell us that we are increasing
>>mathematical literacy.[1]
Again, I suspect that Alfie has no idea what this recurrence relation is, or how often it pops up in many areas of mathematics. If teachers "prep" students it will certainly be better than what they're doing now.
Alfie, this post was stupid even for you. You should have titled it "Whoever thought that you couldn't make a living through stupidity never met Alfie Kohn"
My reaction to this is that it's a perfectly valid question. This is precisely the type of mental hurdle one needs to clear when attacking a new math topic, when one is a student or when one is on the job.
I'm a software engineer and I often encounter such tasks when learning a topic like information theory or computation theory. The best definitions -- meaning the most concise and most accurate, and those that lend themselves most to real world implementation -- are phrased in such terms.
>>But perhaps you figured out that the test
>>designers are just asking you to add 3 and 5 to
>>get 8, then add 5 and 8 to get 13, then add 8 to
>>13 to get 21, and so on.
I suspect if Alfie actually knew anything he'd have identified this as the Fibonacci sequence, a very well known mathematical construct.
>>A pair of math educators
For those of you who don't know, "math educators" are often people who know squat about math. That's why they keep inventing math programs that make your kids dumber and dumber.
>>The problem simply requires the ability to follow
>>a rule; there is no mathematics in it at all.
Spoken by one who makes his living writing tripe like this.
Mathematicians identify rules. It would be pointless to have rules if no one is to follow them.
Continued...