Scholarship is driven not just by intellectual curiosity and funding availability. It is also about the rewards that come to scholars for their work. Promotion and tenure opportunities are an important part of this equation.
But in this time of great change, weaknesses of traditional practices for evaluating faculty are becoming more evident. The imperative for institutions to adapt is captured in a 2005 draft report [PDF] of the University of California Academic Council’s Special Committee on Scholarly Communication:
Many faculty may fear that they will be penalized for publishing in new venues. The University will be disadvantaged if innovative forms and media of scholarly dissemination are discouraged for no other reason than that they are new. In the current system we believe that the academic personnel process at times may place excessive reliance on the reputation of the venue to the detriment of specific assessment of the work itself. As the variety of venues for scholarly publication widens, all participants in the review process should rededicate themselves to judicious assessment of all faculty research, in whatever venue, and to extend to innovative forms of publication the same careful evaluation of scholarship upon which the University has traditionally relied to assure the quality of its faculty.
Traditionally scholars in most fields have looked to the impact factor of a journal as a signal, not just of the importance of the journal, but also of the research reported there and ultimately the researcher him/herself. Many believe this is an overly crude system and a barrier to both fairness and needed change in scholarly publishing.
New ways to evaluate research impact of individual works are feasible in the digital environment:
Citebase, still in experimental form, is an online tool that looks on the fly at how often individual papers are downloaded and cited.
Faculty of 1000 uses a stable of thousands of specialists to rate the most important new papers in biology and medicine they read each month from some 800 journals.
Journal editors and peer reviewers are, in effect, gatekeepers for professional advancement in the “publish or perish” world of academe. Some journals are adopting what they believe are improved quality control processes:
Biology Direct, an open access journal, offers an alternative to the traditional model of peer review by making the peer review process open rather than anonymous and making the reviewers' reports public. In doing so, it aims to increase the responsibility of the referees and eliminate sources of abuse in the refereeing process.
Atmospheric Chemistry and Physics puts papers through a rapid review process to assure basic scientific and technical quality then makes them available immediately on their website. There papers are subject to interactive public discussion, during which the referees' comments (anonymous or attributed), additional short comments by other members of the scientific community (attributed), and the authors' replies are also published. After the discussion is closed, feedback is used to shape the final version of the paper.
The new digital scholarship lends itself to experimentation that will better position scholarship to thrive in an increasingly complex world.
Discover ways to make change work for you -- go to Change & You.