How to evaluate and acknowledge a scientifc journal peer reviewer: a proposed index to measure the performance of reviewers

André Márcio Vieira Messias, Rodrigo Pessoa Cavalcanti Lira, João Marcello Fortes Furtado, Jayter Silva de Paula, Eduardo M. Rocha
2017 Arquivos Brasileiros de Oftalmologia  
The peer review system is a worldwide process employed by the scientific community, and is one of the pillars of the ethical and technical scientific communication (1, 2) . A network of researchers working as a growing team of collaborators in the peer reviewer process improves the quality of papers offered to the readers. Those collaborators are the journal's secret treasure in the publishing competitive field and should be committed with the journal success in its area. A reviewer might be
more » ... eviewer might be acknowledged by qualities that can be documented with numbers, such as time taken in reviewing and the number of manuscript's reviews concluded. Nonetheless, the editors can evaluate the reviewer deepness and broadness knowledge in the field of the manuscript and the reviewer clarity and capacity to propose constructive suggestions to the authors. Both variables, time and number, can be assessed by the editorial tools of several internet-based managing platforms, and the scientific quality of the review can be graded by means of a numeric score provided by the editor. Arquivos Brasileiros de Oftalmologia (ABO) adopted the ScholarOne template, where the score can go from grade 1 (minimum quality) to 3 (maximum quality). Beyond those, there are reviewer's qualities that are not detectable in the everyday work, but can be observed in their long-term work and relationship to the journal, such as exemption and dedication to science. The effort to properly credit the reviewers is an increasing concern of several journals and has resulted in the recent creation of the Publons (, a website that lists the completed reviews of their members, after crosschecking the database of the journal members (3) . It can be used to confirm the work done by reviewers worldwide and to find proper reviewers for each scientific topic. Most scientific journals have faced two global challenges in the editorial process: a) to properly acknowledge their reviewers and reward the most dedicated, and b) to recruit and to teach new members, especially with the growing number of peer-reviewed journals available. The challenge is aggravated by the fact that peer review is an example of pro bono work and there is a huge imbalance between the number of scientific reviews performed and the number of reviewers really committed with the review system. A recent published estimate revealed that 5% of the researchers responded for almost 30% of the reviews performed in 2015 (4) . To overcome those challenges, the ABO has provided, for a long time, workshops in the ophthalmology meetings, encouraging mentors to invite new members to the crew. Every year, the list of reviewers that contributed to the ABO is published in its last issue. The most dedicated ones are recommended by their peers and editors to be part of the Editorial Board. Recently, our editorial board strived for the recognition of the best among all ABO reviewers, and to do that; the editorial board accepted the suggestion of Dr. André M. V. Messias, M.D., PhD, in creating an index that considers the number of reviews performed, their quality and the time taken to submit those reviews. It takes into account measurable reviewing variables calculated using the outputs of the manuscript submission platform, as follows: RI = n x s 3 (T/7 + 5) where "RI" is the Reviewer Index; "n" is the number of reviews completed; "s" is the mean score given by the editor, and T is time spend from review acceptance and submission. Feeding data can be obtained from the tool provided by the platforms ScholarOne and Scielo to the ABO used to perform the editorial process of the manuscripts (, where the time-length (in days)
doi:10.5935/0004-2749.20170084 pmid:29267566 fatcat:lghqhvmn6jdxrewad5uk6vtqoa