Summary of the CIHR Foundation Scheme Peer Review Manual [article]

Joanne-Director Simala-Grant
2014
DRAFT August 24, 2014 |5 Key Points:  Reviewers will not see the final ranked lists of other reviewers.  CIHR will calculate a consolidated ranking for each application and list all applications in the competition from highest to lowest.  Each application will have five reviews, a ranking from each reviewer, a consolidated ranking, and a standard deviation.  CIHR will determine which applicants will be invited to submit a Stage 2 application based on the number of applications, consolidated
more » ... ranking, standard deviation, and funds available. Stage 2 focuses on the Quality of the Program and the Quality of the Expertise, Experience and Resources. Specific adjudication criteria are outlined in the table on page 9 of the Foundation Scheme Peer Review Manual. Key Points:  Each application will be assigned to five reviewers, with matching of application content to reviewer expertise.  Reviewers will access their assigned applications and complete their reviews remotely through ResearchNet.  Reviewers will provide a rating for each sub-section of the application (Research Concept 20%, Research Approach 20%, Expertise 20%, Mentorship and Training 20%, Quality of Support Environment 20%).  Reviewers will justify each rating by briefly stating the strengths and weaknesses for each sub-section. Justifications will be used by reviewers in Stage 3 to adjudication applications.  As for Stage 1, reviewers will take into consideration the career stage, research field and institution setting of all applicants when assessing each criterion.  The same adjudication scale will be used by reviewers for Stage 2 as for Stage 1 (Table 1 , above). Guidelines to aid reviewers in interpreting each adjudication criterion for Stage 2 are detailed in the Interpretation Guidelines: Foundation Scheme Adjudication Criteria. Budget Assessment Reviewers will be required to review the requested budget and justification. Key Points:  Reviewers will be provided with information on the applicant's open grant research funding history with CIHR.  Reviewers will determine if the requested budget is appropriate to the proposed research, realistic and well-justified.  Reviewers will assess the appropriateness of justification for requests that are higher than the applicant's historical grant levels.  The budget assessment must not be factored into the scientific assessment. Asynchronous Online Discussion After a reviewer submits a preliminary review, they will be able to see reviews by the other reviewers of an application and discuss that application online in ResearchNet. The asynchronous online discussion for Stage 2 will operate as for Stage 1 (above). If any reviewer has suggested a change in the budget, that should also be discussed at this point. Details of operation for the asynchronous discussion in Stage 2 review are provided in the full Foundation Scheme Peer Review Manual on page 14. Submit the Final Rank List to CIHR When asynchronous discussions are complete, each reviewer will finalize their reviews and their ranked list for all the applications that they reviewed. Final reviews and the final ranked list will be submitted to CIHR. Key Points:  Reviewers will not see the final ranked lists of other reviewers.  CIHR will calculate a consolidated ranking for each application and list all applications in the competition from highest to lowest.  Each application will have five reviews from Stage 1, five reviews from Stage 2, a ranking from each reviewer, a consolidated ranking, and a standard deviation for Stage 2.  CIHR will determine which applicants will be discussed at Stage 3 based on the number of applications, consolidated ranking, standard deviation, and funds available.  Based on the results from Stage 2, CIHR will place the highest ranked applications to be considered for funding in the "green zone" and will not discuss them further.  In the "grey zone", CIHR will place applications that ranked highly but had large standard deviations due to discrepancies in reviewer ranking.  The bottom ranked applications will not be considered for funding. These applicants will receive a notice of decision after Stage 2.
doi:10.7939/r30p0wq8s fatcat:dix35wo275dnfojzt7echmwtde