"Can you believe this summary statement? They said my grant proposal was too preliminary, not focused, too ambitious, just a descriptive survey, a large fishing expedition, did not have an adequate animal model, did not provide a clear rationale, and was viewed with a low level of enthusiasm. And that's just in the opening summary paragraph! There are four more pages of specific comments. What am I going to do?"
The good news is that everyone has received some of these comments in their grant-proposal critiques. The tough part for the scientist is to determine how to respond so that the proposal is in better shape-- much better shape--when it is resubmitted. This essay will discuss how to evaluate and respond to your review and provide ways to seek the best advice for improving your grant application. We start with the role of the reviewer because your understanding of the reviewer's objectives will provide you with a reference point for your response and resubmission. The discussion that follows uses the National Institutes of Health (NIH) system as example, as this is the largest source of biomedical research funding. But the general advice applies to the peer-review systems of other federal agencies and those of private foundations.
Grant Proposal Reviewers
While you may be viewing your grant application as the magnum opus of your life's ambitions and plans--for the next 5 years anyway--a reviewer sees it as one of six to 12 other "magnum opii" projects to evaluate. The reviewer is asked to rank the assigned stack of grants from best to least using his or her experience, and score each of the grants as outstanding (1.0) to not satisfactory (5.0). The reviewer's score allows others in the peer-review group (study section) to gauge his or her level of enthusiasm for the project. Because very few of the grants discussed during the study section meeting will be funded in this submission cycle, each reviewer will choose one or two grants to champion. The reviewer uses his or her knowledge of science to decide if your grant is novel, is likely to result in new, important information, and can be accomplished with your experimental plan.
Notice that we referred above to the reviewer's knowledge of "science" instead of a specific "field." This is because reviewers are often asked to evaluate grants that are not in their specific field. This means that your grant must be written so that any competent scientist working in your general scientific area can evaluate (favorably!) the application. So if there was a misunderstanding about the significance of your project when it was reviewed the first time, on resubmission you need to do a better job communicating and selling the concept and its aims to your audience. This is really the most challenging aspect of grantsmanship: creating an original idea and presenting it so that others want to see your idea(s) tested.
Reading the Critique
After you calm down, you need to face the prospect of resubmitting your proposal. The first step is to go back and carefully read and reread the critique. After the second or third read-through, divide the critique up into the types of action you need to take to improve the grant. Major categories for organizing your approach include:
Rationale for the experiment Preliminary data/proof of principle Experimental approaches and their alternatives Reconfiguring/jettisoning an aim Data interpretation
Rationale for the experiment
Preliminary data/proof of principle
Experimental approaches and their alternatives
Reconfiguring/jettisoning an aim
As you are doing this, you will have ideas on how you want to respond. Jot them down. When this process is complete, call your program director/program officer and seek his or her advice on dealing the review.
Program Officers/Program Directors
NIH program officers, or program directors as they are sometimes called, serve as scientific liaison between the study section that reviews your grant and the institute that will hopefully be funding your work. These people are YOUR representatives, the ones who can push for your grant to get funded. They are there to discuss your grant and to help you improve your chances of getting funded. Ask your program officer if his or her institute has a program in place to help junior faculty prepare their grant applications (and resubmissions) and how you can sign up for this assistance. [Editors note: also check out our "Toolkit" piece, Getting an NIH R01 Grant.]
Many NIH institutes provide on their Web sites detailed information to help new investigators understand the grant review process and improve their grant writing skills. The table below lists some of these Web addresses.
Grant Writing Assistance Sites
NIH Office of Extramural Research
This links to nine of the best NIH Web sites that provide extensive "Grant Writing Tips"
National Institute for General Medical Science
"Tips for New NIH Grant Applicants"
National Cancer Institute
Includes access to book-length PDF for "Everything You Wanted to Know about the NCI Grants Process But Were Afraid to Ask"
National Institute of Neurological Disorders and Stroke
Very thorough "How to ..." guide that includes dealing with an unfundable score
National Science Foundation
NSF Grant Proposal Guide
Each institute sends a representative program officer to observe the study section proceedings and take notes about what happens during the review of each grant. If you are lucky, your own program officer was at the meeting when your grant was discussed. He or she may be able to provide some additional comments for you from the discussion of your grant that was not included in the summary statement. He or she may have read through your grant as well and may have some separate suggestions or comments. The key to talking to your program officer is to listen to what he or she says and pay close attention to suggestions about how to improve your grant application. One of the topics that may come up is whether your grant was sent to the most appropriate study section.
Changing Study Sections
Certain study sections--including, perhaps, the one your proposal ended up in--will NEVER fund the type of science that you do, regardless of its importance, impact, novelty, or creativity, because your topic is just not what that study section "goes for." If this is the case, then you may want to ask your program officer about moving your proposal to a different review group. Although this change can be difficult to accomplish, it is not impossible.
You may be able to make the case that the original study section's expertise may not coincide with the project/field that you are working in. To prove that point to the program officer you must know the expertise of the panel. If you don't know all the members, scientific interests can be quickly obtained by searching the member's publication history using PubMed. If you do ask to switch study sections, then you need to have another one in mind. You can use the NIH Web site to get a listing of the goals of each of the study sections, the types of science they evaluate, and their membership rosters. But even if you do decide to seek a change of study sections, take the criticisms of the old study section seriously and do your best to address them.
Seeking Advice From Colleagues
Now that you have analyzed your critique and talked with your program officer, you should have the beginnings of a plan for resubmission. To cement your plan, you will need additional advice from senior colleagues. Ask several colleagues who have study section experience to review your grant. Give them lots of time to read your reviews and the grant. Did we say lots of time? YES! When you meet with them listen to what they are saying and avoid being defensive or argumentative. If they provide only grammatical comments, take those, but make it clear that you are also interested in scientific and other substantive comments.
Responding to the Critique
For most grants, a resubmission is allowed some form of rebuttal or introduction. In this section, DO NOT attack the reviewers. This will not help you. Begin your response by summarizing the strengths and weaknesses offered in the reviews. Then describe briefly how you have responded to these comments and, in the cases of major changes, indicate how they resolve issues raised by the reviewers.
After these opening one-to-three paragraphs, proceed to specifics using one of two strategies. The first strategy is to deal with each of the reviewers' comments separately. Most people either quote directly or paraphrase the reviewers' criticism, and then provide their response. The second method is to deal with the comments by specific aim, again quoting the reviewers' statements and your changes/responses. This may save space if some of the reviewers' concerns are similar. There usually is not enough space to address each concern; so address the major issues and as many minor comments as you can. You must be concise. This is the first section that the reviewer will read before they launch into reading your proposal, and it sets the stage for the rest of your presentation. Be upbeat!
Specific responses to the critique will vary depending on the category of the reviewers' concerns. Any problems the reviewers have with your rationale/significance must be addressed early in the rebuttal and in the body of the grant application with sufficient clarity that the reviewer will be convinced. Remember: Your goal is not to win the debate on facts, but to win the advocacy of reviewers and study section members.
Sometimes a reviewer will really hate one of your specific aims. You can deal with this in a couple of different ways. The first is to jettison the aim altogether, which will be the end of this problem. If the proposal is still strong without this aim, don't replace it; you may replace it with a new aim that they also hate. Sometimes it is best to take one of the other aims--one the reviewers liked--split it in two, and develop each further. This is a good approach if you can provide each of the revised aims with clear rationale and preliminary supporting data. A second approach is to take the best part of the hated aim and interweave it into one of the other aims.
If you now have the key experiments to fully justify and support your investigation described in the hated aim, then presenting this information and rationale could make the aim fly. If this is the case, be sure to get several colleagues to read both the reviews and the revised aim to see if it does fly, instead of crashing and burning up your grant proposal.
Often reviewers state that they do not understand why you are taking the approach that you proposed when it could be done another way, which they proceed to describe. If this was an approach you hadn't thought of--and one that is likely to work--you should incorporate these suggestions into your plan, either as a main approach or as an alternative to your methods. If you stick with your original plan as the main approach or you do not incorporate the suggested experiments, be sure to justify your rationale for their omission. Make sure your senior colleagues agree with you.
If you lack preliminary data to prove that your system or model is valid, then you need to generate the data before you resubmit. This may include developing specialty reagents such as antibodies, mice, or proteins. As expensive and labor-intensive as some of these experiments may be, you must get this accomplished or your grant resubmission will not be successful.
Data-interpretation issues can come in many parts of the grant review. Problems arise when the reviewer does not believe the model you are testing or the conclusions that you are drawing from your work or that of others. At the very least, you will need to acknowledge that there are other interpretations and that you will consider these alternatives in evaluating your data. You may have to propose experiments to distinguish between the reviewer's view and yours. If the difference in interpretation arises from your experiments, then you will need to provide additional data to back up your claims. Sometimes this problem can be solved with a literature search; sometimes it will require some bench work.
Consultants and Collaborators
Reviewers may question your expertise on a procedure or an experimental approach. Whereas established lab groups are not often challenged on standard procedures, fledgling groups are. Identifying a qualified collaborator or consultant to provide you with reagents, protocols, and the all-important letter of agreement to assist in the project will aid your case. Even established investigators do this. It removes a criticism and it will probably be cost-free.
This Round or Next?
If you cannot provide the necessary data to address the concerns of the study section by the next deadline, you are better off waiting to resubmit until a future round. But what if your Chair is breathing down your neck for you to get your grant in sooner than you are ready? Before you approach your Chair to discuss your rationale, you should have your plan fully developed and on paper. Lay out a written plan that describes what you have to accomplish before the grant application will fly. Indicate the names of your colleagues who have told you that you need to do A, B, and C, and provide a timeline for getting these tasks accomplished. This should show your Chair that you are taking this seriously and that you are listening to the senior faculty--including him or her.
I Addressed All the Concerns, So My Grant Will Be Funded, Right?
To answer this question, there are several points to discuss. The first is the difference between funded and an improved score. Sometimes, the funding level changes such that a fundable score in one fiscal year is no longer fundable in the current year. In this case, a resubmission with minor changes may boost your grant above the pay line.
The second point reflects whether or not you truly did satisfy all the concerns raised by the reviewers, and whether they really had a high level of enthusiasm for the project. If there was a high level of enthusiasm for the project and the issues were addressed, it is likely that the score will improve substantially. However, if there is not a high level of enthusiasm for the project, it is possible that additional concerns will crop up, even if they weren't mentioned during the first review. Although this is unfair in many ways, it speaks to problems with the question(s) being asked or hypotheses being tested. If this occurs, under the advisement of your senior colleagues, you may consider refocusing some of the aims/goals of your proposal. Or, as mentioned above, seek another study section.
Getting an Early Start
As hard as it may be to do, get in your grant application well before the deadline, and make sure the penultimate draft is read by at least one of your senior colleagues.
Although the advice offered here appears to be simple, it is often difficult to perform. Our pride and our attitudes get in the way in handling critiques of our work. The key is to read your critique thoroughly and dispassionately, and respond sensibly to the queries and concerns of the reviewers. Ask your program officer and colleagues for their advice and incorporate these points into your revised application. Stay optimistic and persistent through the process. Ultimately, you will prevail if your science is good and your plan is sound. Gook luck!
Jeremy M. Boss, Ph.D.
Professor of Microbiology and Immunology, Emory University School of Medicine
Susan H. Eckert, Ph.D.
Associate Dean for Finance and Research Administration, Emory University School of Nursing
Authors of Academic Scientists at Work: Navigating the Biomedical Research Career. Available at www.wkap.nl/prod/b/0-306-47493-X