On Aug. 17, the newsmagazine U.S. News and World Report released its yearly rankings of the best American colleges and universities, as high school seniors prepare for a fall devoted to the college application process. Among Northwest schools, the University of Washington in Seattle was the standout, placing 42nd in the National Universities category (11th among public universities). As many critics have argued over the years, however, the task of ranking universities is not nearly so straightforward as U.S. News claims, and the quality of the UW and other Northwest schools is a much more complex issue than the rankings indicate.

Though the UW was the top-ranked Northwest school among large universities, many smaller schools fared well in other rankings, among them Gonzaga University in Spokane, Whitman College in Walla Walla, Wash., and Reed College in Portland. In the western division of Master's Universities (a category dominated by Catholic schools), Gonzaga ranked third. In the Liberal Arts Colleges category, Whitman ranked 37th, Reed 54th. Northwest public universities other than the UW placed far lower: among national universities, the University of Oregon in Eugene placed 112th, Washington State University in Pullman 118th. Oregon State University in Corvallis was ranked as a Tier 3 school, placing it behind more than half of the national universities.

The rankings, which have been released yearly since 1983, are a cash cow for U.S. News and World Report and generate a great deal of media attention. Much of the higher education community, however, looks upon them with distaste. The rankings are based on a variety of factors, including peer assessment, SAT scores of students, endowment size, and alumni giving, that tell a great deal about prestige and student expectations but very little about the quality of education. Many colleges, including top-ranked institutions, have complained about the rankings, saying that they do not provide any real measure of academic quality and that they provide incentives for colleges to game the system.

Other critics have accused the magazine of constantly adjusting its formula in order to ensure large changes in the list and thereby garner more attention. A 2000 article in Washington Monthly, a center-left political magazine, reported that in the early years of U.S. News' statistical analysis, the process was carefully managed to ensure that Harvard, Princeton, and Yale would take the top spots. In June, the Annapolis Group, a consortium of liberal arts colleges, announced the creation of a study group to examine how to provide more accurate and relevant information about their academic programs to prospective students.

Among the key complaints of universities, which were echoed in an internal review commissioned by U.S. News a decade ago but never satisfactorily addressed, is the failure of the rankings to judge academic quality. The magazine claims that some of the scores it assigns — for faculty pay, percentage of classes with small sizes, and student selectivity — do point toward academic quality. These are at best rough and uneven indicators, and have never calmed the critical uproar.

One of the most consistent and prominent detractors of U.S. News and World Report's rankings and methodology has been Reed College, which for more than a decade has refused to participate in the peer review or send its statistics to the magazine. U.S. News has continued to rank the college, relying on outside information. In a 2005 article published in Atlantic Monthly titled "Is There Life After Rankings?", Reed president Colin Diver discussed his experience at a college that opted out of the rankings game, noting that the peer review portion was essentially meaningless and that the rankings had introduced an incentive for colleges to follow practices at odds with their educational missions in order to move up a few places.

Few schools have followed Reed's suit, fearing that their position on the rankings will be placed in jeopardy, thus weakening their prestige with prospective students and alumni donors. Reed did indeed fall in the rankings; after it first declined to provide statistics, U.S. News gave it the lowest possible score in each category, sending it plummeting from the second quartile to the fourth. (In response to angry claims that U.S. News did this as reprisal, the magazine began using information from independent sources, and Reed has returned to the top tier of liberal arts colleges.) Many university administrators feel that for them to end compliance with the magazine, there would have to be a critical mass of schools leaving together. Reed has thrived since beginning its policy of non-compliance, though other colleges attribute this to Reed's iconoclasm and reputation for intellectually challenging courses.

There have been numerous other attempts at ranking universities. The Princeton Review, a test prep agency, releases a compendium of ratings based on student surveys. Some other rankings have focused on the amount of research that various universities produce. Perhaps the best-known example is the yearly rankings done by academics at China's Shanghai Jiao Tong University (SJTU), who base their figures on an index identifying the number of faculty members with Nobel Prizes and Fields Medals as well as numbers of articles published in high-profile, peer-reviewed academic journals (rankings are slightly modified according to size of institution).

The SJTU rankings have been touted, and perhaps oversold, by The Economist, the British-based, market-oriented newsmagazine, which noted that American universities dominated the rankings. The UW has ranked consistently in the top 20, coming in at number 16 in 2007. As with other measures, these rankings do not necessarily reveal much about the quality of academic life for students. While research production does provide some measure of professors' ability to work at the cutting edge of their fields, it has nothing to do with their dedication to instructing students. Thus the UW's high ranking is indicative of its success at churning out research — always its strongest suit — but has little to do with its quality as an educational institution.

Rankings such as these can tell quite a lot about a university, but not nearly as much as their authors seem to believe. Though the U.S. News and World Report rankings make a half-hearted attempt at gauging academic value, the main indicators primarily track institutional prestige and student expectations. Meanwhile, the SJTU rankings, which purport to measure the world's best universities, merely identify the institutions doing the most research. Since the UW is a massive institution and a perennial leader among recipients of federal grants, it comes as no surprise that it fares well in rankings that are based off of research production. Since it is also a public university with a clear social and educational mission, it also comes as no surprise that it has neither the prestige nor the ability to draw from only the most elite students that many private universities do. In the end, as Reed's Colin Diver points out, the quality of a university as an educational institution is a complex issue, depending to a great degree on how well an institution matches its educational mission with the needs and goals of its students.