Today Geoff Davis, creator of PhDs.org, and Peter Fiske, motivational speaker and columnist for Science's Next Wave, announced that they will not be ranking graduate departments based on the anonymous online survey that they conducted from 25 April to 8 July 1999.
One of the primary goals of the survey was to rank graduate school departments based on student satisfaction. And although the American Association for the Advancement of Science (publisher of Next Wave), the American Chemical Society (ACS), and the National Association of Graduate and Professional Students (NAGPS) all helped to publicize the survey, Davis and Fiske recruited most of the voluntary participants through "viral marketing"--the Internet industry's term for the old "he told two friends, she told two friends" approach to information dissemination.
This strategy generated more than 6500 responses, but they weren't uniformly distributed among departments. "We tried to lower the barriers to responding," says Fiske, "but there are still lots of departments with only one response."
The spotty response rate forced the two part-time social scientists to temporarily abandon one of their primary goals: ranking departments. "Several deans were upset about being ranked with such a small sample," says Fiske, who notes that releasing the rankings prematurely would enable people "to reject it outright as biased and irresponsible." But he notes that the survey will continue to gather data "forever," which in time will increase the sample size, making the point moot.
Fiske and Davis are, however, considering a limited release of the rankings. Although a firm decision has not yet been made, Davis tells Next Wave that the tentative rankings will probably be distributed to graduate school deans. "They need to see this information," he says. And Fiske speculates that seeing the preliminary rankings will put a bit of pressure on department heads to start addressing student concerns.
Rick Cherwitz, associate dean of the Graduate School of the University of Texas (UT), Austin, agrees with Davis's and Fiske's decision: "This study is really important, and it adds a whole new dimension" to our picture of the graduate school experience. But "I think it is a good idea to withhold the rankings for now," he adds. "It will encourage people to look at all the variables in the study."
In addition to withholding the rankings, Fiske and Davis have also decided to withhold narrative comments collected in the survey. "Maybe people weren't thinking about the future when they wrote in," says Davis, "but they said some things that were not appropriate for wide release." But as with the rankings, Davis and Fiske plan to distribute the comments to the relevant departments and graduate schools. Graduate school deans and department chairs applaud this decision, noting that the comments could become the most valuable part of the survey by helping them identify problems and correct misconceptions. "By reading comments from different departments, I may be able to see systemic problems," says UT Austin Graduate Dean Teresa Sullivan. And University of Hawaii zoology department chair Andrew Taylor has already used the comments to discover a "mistaken rumor among the students" in his program.
Despite these specific concerns, students, faculty, and administrators all agree that the survey is a useful new tool in their efforts to improve the lives of graduate students. "Too many students are faced with hostile environments, and it's making it hard for us to get work done," says Susan Mahan-Nieber, a physics graduate student at Washington University in St. Louis and former president of the NAGPS. "This survey gives a realistic picture of how each department is working to improve the experience for graduate students."
With these results Sullivan and other administrators can now begin the process of addressing what needs to be improved at their institutions. "Although it is hubris to think I can fix everything," she says, "I will try to find two or three factors [in the survey] where I can really get some leverage" on the problems facing grad students.