For years, state legislators, parents, and even his own boss had been hectoring Frank Ashley, the vice chancellor of academic affairs for the Texas A&M University System, to tell them whether his highest paid professors were worth their often fat paychecks.
Ashley responded with a spreadsheet that listed each of his faculty members according to how much money they made or lost for the university.
The study calculated an individual professor's "revenue" based on the tuition he or she brought to the school -- a product of the number of students taught -- and the amount of research awards and grants he or she obtained, among other factors. The greater the number of classes and students taught, the greater the revenue. If a professor's annual salary was lower than the amount of revenue generated, it was black. Otherwise, it was red.
Of the 50 highest compensated faculty members, only five appeared to be in the black and earning their keep. The rest were crimson.
Although the study, still in draft-mode, was meant for internal use only and, by Ashley's own admission, deeply flawed, the Texas A&M University System was forced to release the 265-page document on its own website after The Eagle, a local newspaper, filed an open records request.
The data revealed, for example, that while one faculty member at Texas A&M earned more than $500,000 each year, the average counterpart at its College Station campus made around $120,000.
The Texas A&M University System removed the spreadsheet from its website within days, after discovering that much of the data was found to be inaccurate. It is no longer available. (Click here to download the spreadsheet, provided courtesy of the Center for College Affordability and Productivity.)
Ashley's spreadsheet was, of course, a touchstone for many of the anxieties that riddle the ongoing debate around higher education reform. Academia is clubby, professors' performances hard to gauge, and parents and students can easily become financially indentured in their quest for a small piece of the American Dream.
And in Texas, like everywhere else, these issues have hardly disappeared. Similar to most states, Texas has weathered enormous budget cuts. Texas A&M, like other public institutions, has been especially hard hit, with as much as 15 percent, or $32 million, sliced from its budget in recent years. According to Jason Cook, Texas A&M's spokesman, the university stands to lose an additional $53 million in state funds over the next two years.
With the university forced to further cinch an already tight belt, Ashley, as if on cue, is at it again. Despite his detractors, he has assembled a second appraisal of professor productivity, using data from the current academic year. It's a move he predicts other schools, particularly in Texas, are likely to follow.
"With the economy where it is right now, everyone wants to make sure they're getting the most out of what they're paying for," says Ashley, who spent much of the last month finalizing the data, double and triple checking for any miscalculations. Of prior criticism, Ashley is the first to admit that the original spreadsheet was nothing if not a work in progress. He's hoping for a second shot at redemption.
Its fate now rests with the school's 10-person board of regents, who could opt to make it public. Or, as was the case last fall, an open records request may force its release. Recently elected board chairman Richard A. Box calls such an outcome "almost inescapable, since anyone can request it and the information exists in the public domain."
Whether a matter of days or weeks until the information is again made public, Ashley holds firm. "There's this widespread belief that the professoriate is a nice job," he says. "We want to make sure that we're being transparent and good stewards of the money that taxpayers provide for us."
LIES, DAMN LIES, AND STATISTICS
Texas Governor Rick Perry and the Texas Public Policy Foundation, an Austin-based conservative think tank, have come out in favor of Ashley's metric.
But University of Texas President William Powers Jr. -- A&M's arch rival -- remains unconvinced Ashley has proven anything. He's also opposed to the use of such bean counting at his Austin campus.
He worries that A&M's calculations undermine a university's dual mission of teaching and research, which, by its very nature, tends to not only be time consuming but expensive and difficult to quantify.
"Yes, we are cutting budgets and resources are scarce. And yes, we need to be productive and efficient in all that we do and we are in favor of doing things in more cost-effective ways," says Powers. "But we need to do it in a way that recognizes how research is done."
As an example of how hard it can be to quantify the value of research, Powers cites University of Texas Professor John B. Goodenough's discovery of the materials that make lithium-ion rechargeable batteries, which hinged on the decade-long, dedicated scholarship of his colleagues -- work that appeared profitless for a very long time.
Meanwhile, the push for openness at the college and university levels continues.
"Internally, colleges have been conducting these kinds of assessments for decades. The part that's new, the part that's novel is the public disclosure, the transparency," says Richard Arum, a professor of sociology and education at New York University. "Whether we like it or not, we're likely to see more of these measures."
Colleges and universities use different metrics to evaluate their faculty, but there aren't uniform standards for assessing performance and compensation. A range of data may be gathered, including student evaluations, peer evaluations, research grants and awards and publications. Some schools place more value on teaching, while others emphasize research. A professor's stature and reputation also influence compensation, further muddying the issue.
While some professors may bristle at the suggestion, evaluation is nothing new for K-12 teachers. Nationwide, test score data is used internally to determine factors like teacher compensation, bonuses and even who gets fired. More recently, administrators have used value-added analysis, a tool that tries to determine the value a teacher adds to a student's individual achievement, to gauge a teacher's effectiveness.
Yet unlike assessments in K-12 that rely on standardized tests, measuring learning outcomes is more difficult to quantify in higher education. Students are exposed to numerous faculty, making it difficult to tie particular academic gains or losses to an individual professor.
"With this desire for accountability comes the issue of how to measure it," says Scott E. Carrell, an economist at the University of California, Davis. Carrell notes that in the absence of standardized tests, learning gains can be difficult to measure at colleges and universities where students choose their own courses and professors write and grade their own exams.
"Such a system makes it impossible to credibly measure professor quality," Carrell observes.
Recently, Carrell and his colleague, James E. West, averted the difficulty of measuring student outcomes by using the U.S. Air Force Academy as their laboratory. Students at the Air Force Academy take the same classes and follow a prescribed curriculum, distinguishing them from other college students and allowing for a great, controlled experiment.
Published last June in the Journal of Political Economy, the findings showed that neither final test scores nor student-authored class critiques are very good predictors of a professor's effectiveness. The study, which didn't examine compensation, did conclude that students learned more from full professors than from adjuncts.
All of which leads to an interesting question: If a faculty member's effectiveness can't yet be measured at most schools, do we subject what we can calculate -- say, a professor's compensation -- to greater scrutiny?
THE TENURE TRACK
According to the College and University Professional Association for Human Resources, full professors at four-year colleges and universities on average earn between $70,000 and $100,000. Business professors reside at the high end of the pay scale; theologians rank among the lowest. And for the second year in a row, faculty members at public colleges have not seen an increase in pay. Meanwhile, those teaching at private colleges have seen an average raise of only two percent.
Still, Andrew Hacker, who's taught political science at Queens College for more than 40 years, is troubled by the compensation rates for those at the superstar end of the academic spectrum. Last year, he and Claudia Dreifus co-wrote the book "Higher Education?: How Colleges Are Wasting Our Money and Failing Our Kids -- and What We Can Do About It."
In it they wonder what a professor at Yale University, who makes $817 an hour and gets every third year off, does to earn such a high salary. You can hear Hacker describe how they calculated such a high hourly rate in a video interview below:
"Yes, I know they're going to talk about grading papers and committee assignments, but I've seen enough of the top professors to know they never change their lectures, their graduate students do all their grading, and the committee stuff is a waste of time, anyway," he says.
Hacker isn't alone. "We see an enormous variation in what people do and what they get paid to do it," says Richard Vedder, who directs the Center for College Affordability and Productivity in Washington, D.C. and is a professor of economics at Ohio University.
"Money gets dropped out of airplanes over our campuses, alumni are nostalgic about their alma maters, legislators want to provide economic access -- and there's no accountability," he adds.
Vedder describes Texas A&M as "a pioneer, whose attempt is not only a noble effort, but one that I predict will spread." Using the Texas A&M data, he and his team of researchers analyzed whether a relationship exists between the amount of money a professor makes and the number of students he or she teaches. The results are mixed.
Matthew Denhart, administrative director of the College Affordability center, conveyed the findings using this calculus: "Professors with tenure make more money; the more a professor teaches the more money he or she makes; and the professors that bring in research money get paid more than those who do not."
Even if such measurements are less than perfect, others say universities will grow more accountable by becoming more transparent. Texas A&M's data release caught the attention of Michael Hout, who teaches sociology at the University of California, Berkeley and studies issues of accountability in higher education. He applauded its goal, saying that "public universities get the desired results but function as 'trust me' systems that exist behind closed doors, where the public can't see what's going on. This is an attempt to change that."
While Hout sees the Texas A&M model as fundamentally flawed, given its reliance on one number attributed to each academic, he also criticizes the University of California system, where assessment is based on a portfolio review and remains entirely hidden from the public.
"The future has got to include something in between, and right now the academic community and other stakeholders are trying to figure out where that middle ground is," said Hout. "If we're lucky, we'll come up with something appropriate in that process."
Some are less convinced. Daniel Hamermesh, an economics professor at the University of Texas at Austin, has been an outspoken critic. He frets that the Texas A&M metric might be used at his institution, and he described its narrowness as absurd and even dangerous: "People love concreteness and it's a very attractive thing, but this measurement is such a narrow aspect of what we do that it misses the overwhelming majority of everything else."
RED AND BLACK
When the Texas A&M University System released Ashley's spreadsheet last fall, Dudley L. Poston, a tenured professor of sociology, found the information too enticing to pass up. He recalled first typing in his own name, followed by the names of his fellow department members. He finally glanced at a dozen or so friends.
"I quickly discovered that I'm teaching more than most faculty, that I have more students than most faculty and that I'm publishing more than most faculty," says Poston. "But when it comes to financial accounting, I'm in the red."
With Poston's salary and estimated benefits totaling $302,545, the spreadsheet said he cost more than he brought in.
This despite teaching 112 students, including 13 Ph.D. students and five Master's students, whose committees he chaired. Ashley's spreadsheet also failed to include a $450,000 grant from the U.S. Department of Agriculture, an omission Poston believes would have landed him squarely in the black.
In order to make the revised calculations more sophisticated and less prone to error, Ashley confirmed individual professor data with each administrative head. The new rubric also includes a more complicated and complete breakdown of different types of tuition revenue.
After the data was publicly released, Poston said it was not uncommon for fellow faculty members in the red to acknowledge and congratulate each other, as if being granted admittance into a secret society. Some have since informally assembled into what they term a "red brigade."
Outside Texas, Poston received correspondence from colleagues in other states. They teased him, suggesting perhaps he wasn't carrying his own weight. They also expressed concern that similar reforms targeting individual faculty members might soon be headed their way.
When the Texas A&M University System releases the second round of Ashley's data later this spring, Robert S. Bednarz, who teaches in the geography department, already expects to be in the red. After 32 years at Texas A&M, he took the second sabbatical of his career last fall, rendering his teaching load nonexistent and causing his student revenue to evaporate.
He describes Ashley's bookkeeping as demoralizing.
"In this age of accountability, we're evaluating people all over the place, but seldom are we willing to take the time to actually evaluate them well."
While Ashley doesn't disagree, he also doesn't see it as a punitive tool for evaluating his faculty members. As a former dean and provost, he understands the rubric is inherently beset by imprecision.
Still, Ashley believes more transparency is better than none at all. "Higher education is very efficient in using the funds we receive by the state," he says, "and it's time we finally showed that."