Crowdsourcing Analysis of Twitter Data on Climate Change: Paid Workers vs. Volunteers

Andrei Kirilenko, Travis Desell, Hany Kim, Svetlana Stepchenkova
<span title="2017-11-03">2017</span> <i title="MDPI AG"> <a target="_blank" rel="noopener" href="" style="color: black;">Sustainability</a> </i> &nbsp;
Web based crowdsourcing has become an important method of environmental data processing. Two alternatives are widely used today by researchers in various fields: paid data processing mediated by for-profit businesses such as Amazon's Mechanical Turk, and volunteer data processing conducted by amateur citizen-scientists. While the first option delivers results much faster, it is not quite clear how it compares with volunteer processing in terms of quality. This study compares volunteer and paid
more &raquo; ... rocessing of social media data originating from climate change discussions on Twitter. The same sample of Twitter messages discussing climate change was offered for processing to the volunteer workers through the Climate Tweet project, and to the paid workers through the Amazon MTurk platform. We found that paid crowdsourcing required the employment of a high redundancy data processing design to obtain quality that was comparable with volunteered processing. Among the methods applied to improve data processing accuracy, limiting the geographical locations of the paid workers appeared the most productive. Conversely, we did not find significant geographical differences in the accuracy of data processed by volunteer workers. We suggest that the main driver of the found pattern is the differences in familiarity of the paid workers with the research topic.
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="">doi:10.3390/su9112019</a> <a target="_blank" rel="external noopener" href="">fatcat:mwdlckspebh63px2y4ruyitbza</a> </span>
<a target="_blank" rel="noopener" href=";attachment=1" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href=""> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> </button> </a>