THE BLOG
09/23/2014 02:43 pm ET Updated Nov 23, 2014

Asking Questions Later

JC559 via Getty Images

People are working very hard to change the admissions policy and standards for New York City's esteemed specialized high schools, including Brooklyn Tech, Bronx Science and Stuyvesant. While a battle is being waged in Albany about changing admissions criteria -- a 1971 law placing responsibility in the hands of the state is being challenged -- the NYC Department of Education (DOE) has now set to work on revising the specialized high school test (SHSAT) itself. Last week a 71-page Request for Proposals (RFP) was posted on the DOE website, laying out extensive parameters for potential test makers to submit fleshed-out proposals in October. October 23 to be exact, 30 days from today.

For changes of this magnitude that affect the educational trajectory of thousands of NYC students, it seems even fundamental questions hadn't been considered when putting the extensive rubric for the proposal together. Questions that must be asked. So, DOE, I thought I'd lend a hand and ask them now:

Is one month really enough time to reimagine a test of this magnitude? Might it be necessary to give developers more time to thoroughly work through the parameters, especially as the purpose of this reboot is to set up an entirely different exam structure, covering significantly different content?

Aren't these significant changes setting up the kids who will take the inaugural test in the fall of 2016 to be guinea pigs?

Is it too late to consult with teachers, administrators, alumni, and parents of the specialized high schools? They're the ones invested in the rigor and curriculum, and who know best what students need to succeed at these rigorous institutions.

The onus is on the testing companies to make sure the tests are proofread and content has been vetted. But that doesn't always work so well. Remember the "Pineapples Have No Sleeves" debacle a few years ago -- an incoherent passage on the ELA exam (my daughter happened to take that specific test) that posed questions with no clear-cut answers?

What if this initial test is a disaster? Will you be able to quietly make it go away, like the six teacher evaluation tests administered last fall and spring that aren't happening again this year?

Are you anticipating the inevitable avalanche of legal challenges from the families of kids who don't get in for fall 2017?

Don't you think introducing a constructed response and/or essay section introduces subjectivity? Bias? Opens the doors to favoritism and/or exclusion?

Where are you going to find qualified individuals to objectively grade 30,000 essays, in the middle of the school year? Will teachers be taken out of classrooms to grade even more tests than they do now, leaving yet more instructional time in the hands of substitutes?

Who's paying for all this extra grading help?

The proposal calls for the test to be translated into multiple languages: Arabic; Bengali; Chinese (both Mandarin and Cantonese); French; Haitian Creole; Korean; Russian; Spanish; and Urdu. This brings up a host of questions which such as:

Who's translating? Will anyone at the DOE be involved in the process? Whose judgment call is it whether all 10 translated versions are at the same difficulty level as English? What happens if, say, the Urdu version doesn't hold up? Will all other versions of the tests need to change?

The proposal calls for a SHSAT test prep program (this is actually a great idea) that would be available to 150,000 students. Will all test prep material be available in all these languages? Won't that increase production costs across the board? Have you thought of a reasonable distribution system to get correct language information to each of kids interested in taking the exam?

Some more:

Will translators be available on test days to help students who have questions? Will proctors who speak all these languages be present at all test sites?

A big one: who's grading the potential essay component when there are now a range of languages involved? Who's setting the standard for these graders? How can any modicum of objectivity be maintained with multiple graders in multiple languages?

An even bigger question: how will kids who took the test in other languages handle the workload, which is primarily in English, in these rigorous schools? Will more English as a Second Language (ELL) classes need to be opened? Will there be an additional budget stream in place to provide extra support for these kids whose English skills aren't up to AP Language and Composition, Physical Anthropology or Forensic Criminology? What becomes of the schools' English departments and the NYS mandated four years of high school English?

Your RFP mentions that field testing is mandatory, stating, "The items should be tested on high ability students residing primarily in urban areas. It is expected that each item administered in these tryouts will have responses from an adequate number of students in each racial/ethnic subgroup (African American, Hispanic, Asian and White), as well as on approximately equal size groups of males and females, in order to statistically detect the presence of differential item functioning with respect to these subgroups." How about ethnic and religious differences or sexual orientation? Where do you draw the line?

You do know there's an achievement gap in the city that by no means solely manifests itself in SHSAT outcomes -- and that by artificially erasing it by forming test groups by race you're not creating a true representation of the city's student body? And that the results of this racially screened test will not likely yield more balanced demographics at the exam schools?

Which leads me to: in the end, who will benefit from this revised test?

Rather than tampering with an exam that is currently purely objective, solely knowledge based, and has worked for decades, wouldn't it be better to use time, energy and resources to level the playing field by bolstering the educational experiences and foundation for all students, pre-K-8, so that the opportunity to succeed on the admissions exam is truly open to all?