Laypeople Can Predict Which Social-Science Studies Will Be Replicated Successfully

Suzanne Hoogeveen, Alexandra Sarafoglou, Eric-Jan Wagenmakers
2020 Advances in Methods and Practices in Psychological Science  
Large-scale collaborative projects recently demonstrated that several key findings from the social-science literature could not be replicated successfully. Here, we assess the extent to which a finding's replication success relates to its intuitive plausibility. Each of 27 high-profile social-science findings was evaluated by 233 people without a Ph.D. in psychology. Results showed that these laypeople predicted replication success with above-chance accuracy (i.e., 59%). In addition, when
more » ... ddition, when participants were informed about the strength of evidence from the original studies, this boosted their prediction performance to 67%. We discuss the prediction patterns and apply signal detection theory to disentangle detection ability from response bias. Our study suggests that laypeople's predictions contain useful information for assessing the probability that a given finding will be replicated successfully.
doi:10.1177/2515245920919667 fatcat:h2q2txy7dbg75gveomof67jjjq