Big data is everywhere in science today, opening new insights in fields from astrophysics to zoology. For all its promise, though, big data also poses a real danger to academic science, warns Jake VanderPlas, a postdoc in astronomy and computer science at the University of Washington, in an illuminating essay on his Pythonic Perambulations blog. Specifically, VanderPlas believes that universities have to change their reward system or risk losing the very people whose expertise makes big-data research possible.
"Where scientific research is concerned, this recently accelerated shift to data-centric science has a dark side, which boils down to this: the skills required to be a successful scientific researcher are increasingly indistinguishable from the skills required to be successful in industry," VanderPlas writes. "While academia, with typical inertia, gradually shifts to accommodate this, the rest of the world has already begun to embrace and reward these skills to a much greater degree. The unfortunate result is that some of the most promising upcoming researchers are finding no place for themselves in the academic community, while the for-profit world of industry stands by with deep pockets and open arms." (Note that the emphasis is in the original.)
Software and the ability to write, test, and maintain it are fundamental to the new style of research, VanderPlas explains, and "the new breed of scientist must be a broadly-trained expert in statistics, in computing, in algorithm-building, in software design, and (perhaps as an afterthought) in domain knowledge as well."
But, he continues, the considerable time needed for successfully "building and documenting [the] software tools" essential to big-data research "is time spent not writing research papers, which are the primary currency of the academic reward structure." For this reason, "those poor souls whose gifts lie in scientific software development rather than the writing of research papers will mostly find themselves on the margins of the academic community."
Industry, however, rewards these skills handsomely—just one of the reasons that this kind of work "is highly attractive: it is addressing interesting and pressing problems; it offers good pay and benefits; it offers a path out of the migratory rat-wheel of temporary postdoctoral positions, and often even encourages research and publication in fundamental topics."
VanderPlas offers a couple of intriguing suggestions for countering industry's allure and keeping the talent needed for big-data research within academe. First, universities should develop "a new standard for tenure-track evaluation criteria: one which considers the creation and maintenance of open software along with more traditional activities like publication and teaching. This will remove a main disincentive against investing time producing clean, well-documented, and open code." Second, universities should create new tenure-track positions that "particularly emphasize and reward the development of open, cross-disciplinary scientific software tools" and attract people "interested in building and maintaining the essential software used by themselves and their colleagues." (Note that the emphasis is in the original.)
He also makes one suggestion that has been mentioned so often in blue-ribbon studies and panel reports that it has become a cliché: Postdocs need and deserve increases in what VanderPlas rightly terms their "absolutely unsustainable" (not to mention unconscionable) pay scale. But he offers a reason that rarely appears in this discussion: "Those with the skills mentioned in this article could easily ask for several times that compensation in a first-year industry job, and would find themselves working on interesting problems in a setting where their computational skills are utilized and valued."
These days, budget-conscious universities often hire nontrack employees rather than tenure-track faculty to fill openings, making VanderPlas's hoped-for new tenure lines unlikely to appear any time soon. And they are accustomed to viewing postdocs as low-paid, highly skilled labor that can be imported as needed. But VanderPlas's astute observations merit serious consideration because, without "changes in the culture of academia itself" like those he suggests, he fears that "the progress of scientific research will be severeley [sic] handicapped in the coming years." You can read his informative essay here.