Filters








3 Hits in 2.3 sec

In Search of Ambiguity: A Three-Stage Workflow Design to Clarify Annotation Guidelines for Crowd Workers [article]

Vivek Krishna Pradhan, Mike Schaekermann, Matthew Lease
2021 arXiv   pre-print
We propose a novel three-stage FIND-RESOLVE-LABEL workflow for crowdsourced annotation to reduce ambiguity in task instructions and thus improve annotation quality.  ...  Results show improved annotation accuracy and further insights regarding effective design for crowdsourced annotation tasks.  ...  Crowdtruth 2.0: Quality metrics for crowdsourcing with disagreement. arXiv preprint arXiv:1808.06080 Egelman, S., Chi, E. H., and Dow, S. (2014). Crowdsourcing in hci research.  ... 
arXiv:2112.02255v1 fatcat:xpvsuclbi5dfrmfm2ejx54qdii

Us vs. Them: A Dataset of Populist Attitudes, News Bias and Emotions [article]

Pere-Lluís Huguet-Cabot and David Abadi and Agneta Fischer and Ekaterina Shutova
2021 arXiv   pre-print
In this paper, we present the new Us vs. Them dataset, consisting of 6861 Reddit comments annotated for populist attitudes and the first large-scale computational models of this phenomenon.  ...  We investigate the relationship between populist mindsets and social groups, as well as a range of emotions typically associated with these.  ...  Crowdtruth 2.0: Quality metrics for crowdsourcing with dis- agreement. Mai ElSherief, Vivek Kulkarni, Dana Nguyen, William Yang Wang, and Elizabeth Belding. 2018.  ... 
arXiv:2101.11956v3 fatcat:csk3irvilnd5jl7io2q56x5ag4

Us vs. Them: A Dataset of Populist Attitudes, News Bias and Emotions

Pere-Lluís Huguet Cabot, David Abadi, Agneta Fischer, Ekaterina Shutova
2021 Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume   unpublished
In this paper, we present the new Us vs. Them dataset, consisting of 6861 Reddit comments annotated for populist attitudes and the first large-scale computational models of this phenomenon.  ...  We investigate the relationship between populist mindsets and social groups, as well as a range of emotions typically associated with these.  ...  CrowdTruth includes a set of metrics to analyse and obtain probabilistic scores from crowdsourced annotations.  ... 
doi:10.18653/v1/2021.eacl-main.165 fatcat:loo7dr255rb6fm7pskezqfugoq