Internet Archive Scholar

Crowd Guilds: Worker-led Reputation and Feedback on Crowdsourcing Platforms [article]

Mark E. Whiting, Dilrukshi Gamage, Snehalkumar S. Gaikwad, Aaron Gilbee, Shirish Goyal, Alipta Ballav, Dinesh Majeti, Nalin Chhibber, Angela Richmond-Fuller, Freddie Vargus, Tejas Seshadri Sarma, Varshine Chandrakanthan (+14 others)
2016 arXiv   pre-print

Preserved Fulltext

fulltext thumbnail
Web Archive Capture PDF (481.4 kB)
https://web.archive.org/web/20200826193245/https://arxiv.org/pdf/1611.01572v2.pdf

A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL. The file type is application/pdf.

Other Versions

2017 Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing - CSCW '17   unpublished
doi:10.1145/2998181.2998234 fatcat:x3roxtg4rnhmzhooyfzwygizei
2016 pre-print
arXiv:1611.01572v1 fatcat:xfkudwopgjfppmiyv2mfwxx6qi
2017 accepted
arXiv:1611.01572v3 fatcat:o3gzx757mnf5fmewemdeetnrda
Crowd workers are distributed and decentralized. While decentralization is designed to utilize independent judgment to promote high-quality results, it paradoxically undercuts behaviors and institutions that are critical to high-quality work. Reputation is one central example: crowdsourcing systems depend on reputation scores from decentralized workers and requesters, but these scores are notoriously inflated and uninformative. In this paper, we draw inspiration from historical worker guilds
more » ... g., in the silk trade) to design and implement crowd guilds: centralized groups of crowd workers who collectively certify each other's quality through double-blind peer assessment. A two-week field experiment compared crowd guilds to a traditional decentralized crowd work model. Crowd guilds produced reputation signals more strongly correlated with ground-truth worker quality than signals available on current crowd working platforms, and more accurate than in the traditional model.
arXiv:1611.01572v2 fatcat:zn2vdnlforahpcsziytvlmlyyy
Multiple Versions
Citation

Mark E. Whiting, Dilrukshi Gamage, Snehalkumar S. Gaikwad, Aaron Gilbee, Shirish Goyal, Alipta Ballav, Dinesh Majeti, Nalin Chhibber, Angela Richmond-Fuller, Freddie Vargus, Tejas Seshadri Sarma, Varshine Chandrakanthan, Teogenes Moura, Mohamed Hashim Salih, Gabriel Bayomi Tinoco Kalejaiye, Adam Ginzberg, Catherine A. Mullings, Yoni Dayan, Kristy Milland, Henrique Orefice, Jeff Regino, Sayna Parsi, Kunz Mainali, Vibhor Sehgal, Sekandar Matin, Akshansh Sinha, Rajan Vaish, Michael S. Bernstein. "Crowd Guilds: Worker-led Reputation and Feedback on Crowdsourcing Platforms." arXiv (2016)