Lies, damn lies, and college rankings
Posted by metaphorical on 13 March 2007
“In other words, in the absence of real data, they will make up a number.”
We know there’s lies, damn lies, and statistics. Sarah Lawrence College is tired of the lies, and they’re not going to participate any more. Naturally, they’re going to be punished. It just stands to reason.
In this scenario, the lying statistics are the SAT scores used by the U.S. News & World Report college rankings. It seems that more and more colleges aren’t using SAT scores in their determinations of which applicants to accept. But in order to not upset their U.S. News ranking, they are making SATs “optional.”
The result is a statistical advantage of the “damn lies” sort. As Inside Higher Ed puts it,
When applicants learn that a college is SAT-optional, it doesn’t take an 800 math score to figure out the statistically wise strategy. If your scores are at or above reported averages, submit them. Otherwise, don’t. Not surprisingly then, many colleges that go SAT-optional experience both a surge in applications and an increase in their SAT averages … and their U.S. News rankings go up.
Sarah Lawrence decided it didn’t want any part of that. In an op-ed piece in Sunday’s Washington Post, the school’s president, Michele Tolela Myers, explained:
Two years ago, we at Sarah Lawrence College decided to stop using SAT scores in our admission process. We didn’t make them optional, as some schools do. We simply told our prospective students not to bother sending them. We determined that the best predictors of success at Sarah Lawrence are high school grades in rigorous college-prep courses, teachers’ recommendations and extensive writing samples. We are a writing-intensive school, and the information produced by SAT scores added little to our ability to predict how a student would do at our college; it did, however, do much to bias admission in favor of those who could afford expensive coaching sessions.
The school is doing just as well as it ever did; incoming students measure up just fine against earlier ones. “Their average high school grades, high school ranks and grades in Advanced Placement courses have not changed,” Myers wrote.
But this principled decision has put us in jeopardy. I was recently informed by the director of data research at U.S. News, the person at the magazine who has a lot to say about how the rankings are computed, that absent students’ SAT scores, the magazine will calculate the college’s ranking by assuming an arbitrary average SAT score of one standard deviation (roughly 200 points) below the average score of our peer group.
In other words, in the absence of real data, they will make up a number. He made clear to me that he believes that schools that do not use SAT scores in their admission process are admitting less capable students and therefore should lose points on their selectivity index. Our experience, of course, tells us otherwise.
One option is to just stop playing the U.S. News game entirely. Sarah Lawrence has considered that. But there’s a stiff price to be paid. Myers says,
several faculty members and deans suggested that perhaps it was time to stop playing ranking roulette and opt out of the survey. A few colleges explore this option each year, but most don’t follow through (Reed College is one of the few exceptions I know of), because, like unilateral disarmament, unilateral withdrawal from the U.S. News ranking system is dangerous.
I don’t know if you’ve seen USN&WR lately. It’s as skinny, compared to its heyday, as the abbreviation is compared to spelling out its whole name. The college ranking gig is one of the few things it has going for itself. According to a Folio article back in 1993, in 1992, “the ad-page battle of the newsweeklies, an annual contest between Time and Newsweek, has yielded a surprise winner – U.S. News & World Report.” By 2004, it was “the struggling bottom player among the big traditional news weeklies.”
I should mention that my alma mater, SUNY’s College at Geneseo, is a winner at the U.S. News game of statistical roulette. Over the course of the 1980s, Geneseo steadily rose so that now it consistently ranks among the top mid-sized colleges, private or public. It happens to have done it the hard way, by creating a tough, humanities and science based core curriculum that was a a magnet that attracted serious, smart students and repelled those who are not.
But no matter how well U.S. News has done by Geneseo, you have to admire Sarah Lawrence for preferring no stats to lying stats. Naturally, no honest deed goes unpunished. Hopefully, though, U.S. News will get theirs, as word spreads that its numbers are thin as its ad pages.