THE BLOG
12/02/2014 01:37 pm ET Updated Feb 01, 2015

The Sorry Story of College Rankings and Ratings

It's football season. The debate continues. Who is number one? Every Monday, we learn how the multiple polls ranking the college football powers evaluate the schools. Arguments ensue immediately across the airways, on campuses, in bar rooms and beyond.

The polls compile coaches' opinions (some always vote their team first), college football media experts' opinions and sophisticated computer algorithms that factor head-to-head competition, final scores, statistical results, strength of schedules and more. The results every week drive fans, columnists and coaches to distraction arguing the weekly rankings for the next several months.

Controversy over which teams were number one or two and would play for the national championship continued to rage after the "Bowl Championship Series" was created in 2000 to decide the national champion on the field and not through the final polls. Nothing changed. The debate raged. The arguments moved to whether number two really should be number three and vice versa, or should an undefeated team from outside one of the five power conferences be given a chance to play for the national title instead of a team from one of those conferences with a loss on its record.

This year, that one game system will be replaced by a mini playoff system. Now we will get to argue about how the fifth ranked team was unfairly placed because they lost one game late in the season or because their early "big" wins against "ranked" teams became degraded by the computers because those teams fell out of the polls with multiple "bad" losses. Arguments about the rankings will still rage, the only changes will be fans of more teams will have higher stakes arguments and there will be one more big money bowl game. So what's the point?

If it is so difficult to achieve any consensus on these rankings of the top 25 or top four of 128 FBS college football teams by all the recognized national experts, six separate computer programs crunching mountains of objective data of definitive competitions, how the heck are a bunch of Washington bureaucrats -- most of whom have no other experience in higher education than having attended one or two colleges as a student -- ever going to rank some 4,000 colleges and universities with a wide variety of missions, incredibly diverse student bodies, hundreds of different academic programs and no objective means of comparing them head to head.

In Division I college football there is one mission: win and be number one. While the athletes are wonderfully diverse ethnically and economically, they are all supported with scholarships, dozens of coaches and trainers, and all are big, strong and athletic males. All are pretty similar. Comparisons and rankings are far easier, but still woefully flawed. Ask any fan whose team is not rated number one.

How then do you measure the quality of teaching and learning between students at a research university of 40,000 students relying on large lecture classes of hundreds of students, against the teaching and learning at a university like mine with no classes of more than 25 students? How do you compare the results of being taught by faculty with Ph.D.s whose first love and best talents are in teaching compared to graduate students who teach as a means to earn a Ph.D.? How do you compare and rank outcomes from one school whose freshmen SAT math and reading scores average 1,450 against a school whose students average the average score of 1,015? How do you factor the differences in persistence for students from families with incomes in excess of $150,000 a year against those with incomes of less than $50,000? How do you calculate the differences between first generation college students from students whose parents and grandparents graduated from college; between those who must work to pay their tuition and those who don't have to; between a 37-year old U.S. Army sergeant fitting school in among deployments to a war zone, service and family, and an 18-year old joining a sorority? How do you compare an anthropology major and an accounting student? How do you differentiate between a student graduating into the family business and a single mom with three young children graduating with a social work degree, or between the salaries of a first-year high school science teacher and a first-year engineer. You can't. And Lord knows the last people who should attempt it are a bunch of Washington bureaucrats whose actions and decisions demonstrate an almost complete lack of awareness of the manifold differences and distinctions in the United States "system" of higher education. The incredible diversity of mission, students, and programs that make it impossible to rank schools is the foundation of the great strength of the best higher education system in the world.

But the federal government invests a lot in our colleges, over 95 percent in the form of loans to students, (not an expense to the taxpayer and government by the way, but a $50 billion annual income source for the federal budget in interest income on the loans repaid in recent years). Still accountability is in order, but please, not rankings.

Schools are categorized now by degrees granted: Associate's, Bachelor's, Master's I and II, Doctoral, Research I and II. Likewise they can be categorized by their student demographics: full-time, part-time, average financial need; diversity factors, etc. The federal Department of Education collects all of this data and more. Means and medians can be calculated by categories: graduation rates; average indebtedness; default rates; maybe even placement rates. Schools could be graded based on their results, maybe even required to improve them in some categories until they reach a particular level for their category of institution and student profile. But please -- no rankings or ratings.

The U.S. government tried ratings in 1911 when there were only 600 colleges and only 3 percent of the population attended college: almost exclusively affluent males. They tried to rate, not rank, 300 of the colleges in several categories. The results were such a mess that President Taft had to bury them with an executive order. Let's not repeat that sorry history. The arguments over who is number one in college football are enough for us all.