Peer Review #2 of "Unprofessional peer reviews disproportionately harm underrepresented groups in STEM (v0.1)" [peer_review]

Z Master
2019 unpublished
Peer reviewed research is paramount to the advancement of science. Ideally, the peer review process is an unbiased, fair assessment of the scientific merit and credibility of a study; however, well-documented biases arise in all methods of peer review. Systemic biases have been shown to directly impact the outcomes of peer review, yet little is known about the downstream impacts of unprofessional reviewer comments that are shared with authors. Methods: In an anonymous survey of international
more » ... of international participants in STEM (science, technology, engineering, and mathematics) fields, we investigated the pervasiveness and author perceptions of longterm implications of receiving of unprofessional comments. Specifically, we assessed authors' perceptions of scientific aptitude, productivity, and career trajectory after receiving an unprofessional peer review. Results: We show that survey respondents across four intersecting categories of gender and race/ethnicity received unprofessional peer review comments equally. However, traditionally underrepresented groups in STEM fields were most likely to perceive negative impacts on scientific aptitude, productivity, and career advancement after receiving an unprofessional peer review. Discussion: Studies show that a negative perception of aptitude leads to lowered self-confidence, shortterm disruptions in success and productivity, and delays in career advancement. Therefore, our results indicate that unprofessional reviews likely have and will continue to perpetuate the gap in STEM fields for traditionally underrepresented groups in the sciences. PeerJ reviewing PDF | Abstract 36 37 Background: Peer reviewed research is paramount to the advancement of science. Ideally, the 38 peer review process is an unbiased, fair assessment of the scientific merit and credibility of a 39 study; however, well-documented biases arise in all methods of peer review. Systemic biases 40 have been shown to directly impact the outcomes of peer review, yet little is known about the 41 downstream impacts of unprofessional reviewer comments that are shared with authors. 42 Methods: In an anonymous survey of international participants in STEM (science, technology, 43 engineering, and mathematics) fields, we investigated the pervasiveness and author perceptions 44 of long-term implications of receiving of unprofessional comments. Specifically, we assessed 45 authors' perceptions of scientific aptitude, productivity, and career trajectory after receiving an 46 unprofessional peer review. 47 Results: We show that survey respondents across four intersecting categories of gender and 48 race/ethnicity received unprofessional peer review comments equally. However, traditionally 49 underrepresented groups in STEM fields were most likely to perceive negative impacts on 50 scientific aptitude, productivity, and career advancement after receiving an unprofessional peer 51 review. 52 Discussion: Studies show that a negative perception of aptitude leads to lowered self-confidence, 53 short-term disruptions in success and productivity, and delays in career advancement. Therefore, 54 our results indicate that unprofessional reviews likely have and will continue to perpetuate the 55 gap in STEM fields for traditionally underrepresented groups in the sciences. Manuscript to be reviewed 74 75 Introduction 76 77 The peer review process is an essential step in protecting the quality and integrity of 78 scientific publications, yet there are many issues that threaten the impartiality of peer review and 79 undermine both the science and the scientists (Kaatz, Gutierrez, and Carnes 2014; Lee et al. 80 2013). A growing body of quantitative evidence shows violations of objectivity and bias in the 81 peer review process for reasons based on author attributes (e.g. language, institutional affiliation, 82 nationality, etc.), author identity (e.g. gender, sexuality), and reviewer perceptions of the field 83 (e.g. territoriality within field, personal gripes with authors, scientific dogma, discontent/distrust 84 of methodological advances) (Lee et al. 2013). The most influential demonstrations of systemic 85 biases within the peer review system have relied on experimental manipulation of author identity 86 or attributes (e.g. Goldberg's 1968 classic study 'Joan' vs 'John'; Goldberg 1968; Wenneras and 87 Wold 1997) or analyses of journal-reported metrics such as number of papers submitted, 88 acceptance rates, length of time spent in review, and reviewer scores (Fox, Burns, and Meyer 89 2016; Fox and Paine 2019; Helmer et al. 2017; Lerback and Hanson 2017) . These studies have 90 focused largely on the inequality of outcomes resulting from inequities in the peer review 91 process. While these studies have been invaluable for uncovering trends and patterns, and 92 increasing awareness of existing biases, they do not specifically assess the content of the reviews 93 (but, see Resnik, Gutierrez-Ford, and Peddada 2008) , the downstream effects that unfair, biased, 94 and ad hominem comments may have on authors, and how these reviewer comments may 95 perpetuate representation gaps in STEM fields. 96 In the traditional peer review process, the content, tone, and thoroughness of a manuscript 97 review is the sole responsibility of the reviewer (the identity of whom is often protected by 98 anonymity), yet the contextualization and distribution of reviews to authors is performed by the 99 assigned (handling) editor at the journal to which the paper was submitted. In this tiered system, 100 journal editors are largely considered responsible for policing reviewer comments and are 101 colloquially referred to as the 'gatekeepers' of peer review. Both reviewers and editors are under 102 considerable time pressures to move manuscripts through peer review, often lack compensation 103 commensurate with time invested, experience heavy workloads, and are subject to inherent 104 biases of their own, which may translate into irrelevant and otherwise unprofessional comments 105 being first written, and then passed along to authors (Resnik and Elmore 2016; Resnik, 106 Gutierrez-Ford, and Peddada 2008).
doi:10.7287/peerj.8247v0.1/reviews/2 fatcat:3d7zg56zlvdlxkjimuhg4umobi