Dear GrantDoctor,

Although a colleague's grant submission was reviewed favorably, funding was denied and minor revisions were recommended. All of the requisite changes were made and resubmitted the following year, but instead of being accepted, its rating dropped considerably. We found that the [review] committee had been reorganized, and not one expert on the original grant submission was called to review the second version. In turn, new comments were made, and these comments indicated that a few of the new committee members were not familiar with related literature. I would be interested in what you would do in this situation.

Dear Reader,

No matter where the application was submitted (you didn't say where your colleague's grant application went), the first thing the applicant should do is contact the program officer at the funding agency and explain concerns about the second review. There is a strict procedure about how to go about making appeals--especially at federal agencies such as the NIH, where you are allowed one submission and then two chances to resubmit.

Many funding agencies are reinventing themselves--going electronic, reducing budget request information, reorganizing review groups, and emphasizing the research's impact and "creative" qualities.

Your colleague's application, if recently submitted to NIH, could have been reviewed by a panel that addressed different criteria or different aspects of the research the second time around.

Your discipline could also have changed during the year, or perhaps the study section or review group received a much larger number of applications the second time.

Unfortunately, inconsistencies in the review process are not uncommon and fluctuations in scoring may occur. It was only three short years ago, for example, that NIH's own long-standing process of peer review came under fire by an NIH subcommittee involved in "Improving Peer Review." Charged with addressing a variety of concerns, including how applications are scored, the subcommittee published its recommendations in May 1996.

One of NIH's concerns was that "scores are generated, calculated, and used as if they represent a higher degree of reliability and precision than they actually do." Program staff--the officers who handle the review process--are then "held hostage" and forced to defend funding decisions based upon these "small and likely meaningless mathematical differences."

NIH isn't alone: Just a few weeks ago, Rita Colwell, the director of the NSF, reminded the academic community (including reviewers) to focus on "impact" and "intellectual merit"--criteria that were introduced two years ago!--when preparing and reviewing research proposals.

"Considerable variability" in how study sections are chaired and presided over was a significant observation, and Keith Yamamoto, who headed the advisory committee, said that when a revised application is reviewed, there is usually at least one new reviewer with a different perspective. Since you say all the panelists who reviewed your colleague's revised application were not in the original group, it is possible that many new perspectives were brought into play.

I have already addressed another reader's concerns about appealing a grant review in this column; check it out for further advice. But I would contact the grants officer at the funding agency, explain your situation, and find out directly how to proceed with an appeal if you remain unsatisfied by the quality of the panel review.

-- The GrantDoctor