06/30/2010 06:42 pm ET Updated May 25, 2011

Why PayScale's Data on Which Colleges Offer Best ROI is Useless

PayScale's latest data showing the return on investment of attending various schools is getting a ton of press.

The Wall Street Journal has an item on the data, and so does The Chronicle of Higher Education. On the surface, it's quite interesting. Payscale looked at the costs of colleges and then compared it with the average earnings of the schools' graduates and calculated an annual return on investment over thirty years. Ivy League grads tend to earn a lot of money so, even though those schools have high sticker prices, they also tended to yield the highest returns on investment.

But there's just one problem (actually two: we'll get to the second one later): In order for Payscale's data on return on investment to be valuable, you have to assume that the only difference between at MIT grad (the college with the best ROI) and a Black Hills State University grad (the college with the lowest ROI) is that one went to MIT and the other went to Black Hills State.

With apologies to Black Hills State grads, that assumption is insane: MIT grads are a lot smarter, more ambitious, and more connected before they start college than the average Black Hills State grad. Saying "Go to MIT so you'll earn more money" is like saying "Sleep in a crib so you'll weigh less." I've done extensive research and found that people who sleep in cribs do in fact weigh, on average, a lot less than people who sleep in beds -- but that doesn't necessarily make it valuable weight loss advice.

Perhaps there is some financial benefit to be derived from attending an elite school; and perhaps there isn't. Payscale makes no effort to determine that. Incidentally, there was one study that did: Princeton economist Alan Krueger and Stacy Dale of the Andrew W. Mellon Foundation found that, on average, students who get into elite colleges but attend less elite colleges earn the same amount of money ten years in as students who attend elite colleges. As Krueger wrote, "Children smart enough to get into elite schools may not need to bother."

Payscale furthers the problem with its methodology by penalizing schools with low graduation rates. Here again, elite schools have high graduation rates at least in part (and maybe completely) because they only admit students with high GPAs, SAT scores, and strong extracurriculars; such students are unlikely to dropout regardless of where they enroll. By giving schools like Harvard and MIT higher ROI numbers because of their low dropout rates, Payscale is almost intentionally compounding the impact of selection bias on its study.

Dropping out of college is a decision, not a passive outcome; it's like saying it's unsafe to live in Mississippi because it's the fattest state in America. If you lay off the fatty foods, you'll be just fine. Similarly, you can live in Colorado, the thinnest state in the union, but if you eat 50 Twinkies for breakfast and don't exercise, the demographics will do nothing to help your life expectancy.

Even if the college you attend does have some impact on whether you'll drop out, Payscale's method of controlling for dropout rates assumes that the college you attend determines 100% whether you'll graduate. That's ludicrous.

The bottom line is this: Your earnings trajectory will be influenced by a multitude of factors, with the impact of the college from which you receive your bachelor's ranging anywhere from zero to very small. Parents: Relax! There are far more important things to worry about than where your kid goes to college. You could start by worrying about the impact excessive student loan debt will have on his life if you decide to overextend yourselves in pursuit of an elite diploma.