Does Evidence from Peers Help Crowd Workers in Assessing Truthfulness?

Jiechen Xu, Lei Han, Shaoyang Fan, Shazia Sadiq, Gianluca Demartini
2022 Companion Proceedings of the Web Conference 2022  
Misinformation has been rapidly spreading online. The current approach to deal with it is deploying expert fact-checkers that follow forensic processes to identify the veracity of statements. Unfortunately, such an approach does not scale well. To deal with this, crowdsourcing has been looked at as an opportunity to complement the work of trained journalists. In this work, we look at the effect of presenting the crowd with evidence from others while judging the veracity of statements. We
more » ... nt various variants of the judgment task design to understand if and how the presented evidence may or may not affect the way of crowd workers judging truthfulness and their performance. Our results show that, in certain cases, the presented evidence may mislead crowd workers who would otherwise be more accurate if judging independently from others. Those who made correct use of the provided evidence, however, could benefit from it and generate better judgments. CCS CONCEPTS • Information systems → Crowdsourcing.
doi:10.1145/3487553.3524236 fatcat:extlnj4oobdmtgnpu4df7i2fgi