8,542 Hits in 4.0 sec

Crowdordering [chapter]

Toshiko Matsui, Yukino Baba, Toshihiro Kamishima, Hisashi Kashima
2014 Lecture Notes in Computer Science  
In this paper, we consider tasks where crowdworkers are asked to arrange multiple items in the correct order.  ...  Several statistical crowdsourcing quality control methods for binary and multinomial questions have been proposed.  ...  Baba was supported by the Funding Program for World-Leading Innovative R&D on Science and Technology (FIRST Program).  ... 
doi:10.1007/978-3-319-06605-9_28 fatcat:cmieesw7kjgs7juyeieo6jeuju

Crowdsourced Data Management: A Survey

Guoliang Li, Jiannan Wang, Yudian Zheng, Michael J. Franklin
2016 IEEE Transactions on Knowledge and Data Engineering  
We identify three important problems in crowdsourced data management. (1) Quality Control: Workers may return noisy or incorrect results so effective techniques are required to achieve high quality; (2  ...  ., the crowd) to apply human computation for such tasks. Thus, crowdsourced data management has become an area of increasing interest in research and industry.  ...  Quality Control The requester can select the quality-control techniques provided by the crowdsourcing platform, or design her/his own quality-control methods.  ... 
doi:10.1109/tkde.2016.2535242 fatcat:sit3comyvra4djqkl4xre3kj24

Engineering Crowdsourced Stream Processing Systems [article]

Muhammad Imran, Ioanna Lykourentzou, Yannick Naudet, Carlos Castillo
2014 arXiv   pre-print
A crowdsourced stream processing system (CSP) is a system that incorporates crowdsourced tasks in the processing of a data stream.  ...  In this paper, we position CSP systems within a broader taxonomy, outline a series of design principles and evaluation metrics, present an extensible framework for their design, and describe several design  ...  Acknowledgements Imran et al. 27 The authors would like to thank Jakob Rogstadius and Ji Kim Lucas for their contributions in the development of AIDR.  ... 
arXiv:1310.5463v3 fatcat:b4t34p7b5jcnzb42mkmtg45zuu

Towards a Reliable and Robust Methodology for Crowd-Based Subjective Quality Assessment of Query-Based Extractive Text Summarization

Neslihan Iskender, Tim Polzehl, Sebastian Möller
2020 International Conference on Language Resources and Evaluation  
To investigate the appropriateness of crowdsourcing for this task, we conduct intensive comparative crowdsourcing and laboratory experiments, evaluating nine extrinsic and intrinsic quality measures on  ...  For the first time, and as a fast, scalable, and cost-effective alternative, we propose micro-task crowdsourcing to evaluate both the intrinsic and extrinsic quality of query-based extractive text summaries  ...  of micro-task crowdsourcing for this task.  ... 
dblp:conf/lrec/IskenderPM20 fatcat:2ih34bsdpre35gfbjom6p3ymxy

Crowdsourcing versus the laboratory: Towards crowd-based linguistic text quality assessment of query-based extractive summarization

Neslihan Iskender, Tim Polzehl, Sebastian Möller
2020 Conference on Digital Curation Technologies  
Curating text manually in order to improve the quality of automatic natural language processing tools can become very time consuming and expensive.  ...  for determining the factors overall quality, grammaticality, non-redundancy, referential clarity, focus, and structure & coherence.  ...  Although crowdsourcing services provide quality control mechanisms, the quality of crowdsourced corpus generation has been repeatedly questioned be-cause of the crowd worker's inaccuracy and the complexity  ... 
dblp:conf/qurator/IskenderP020 fatcat:f34j62wayngbbdxwxu6k2egj5a

Community-based crowdsourcing

Marco Brambilla, Stefano Ceri, Andrea Mauri, Riccardo Volonterio
2014 Proceedings of the 23rd International Conference on World Wide Web - WWW '14 Companion  
features of the crowdsourcing task.  ...  This paper is focused on community-based crowdsourcing applications, i.e. the ability of spawning crowdsourcing tasks upon multiple communities of performers, thus leveraging the peculiar characteristics  ...  Performer annotates some items with tags Classify Performer assigns each item to one or more classes Order Performer reorders the (top n) items in the input list Ins.  ... 
doi:10.1145/2567948.2578835 dblp:conf/www/BrambillaCMV14 fatcat:p5r55arwkjbfbo6ugc2bv6yzh4

Crowdsourcing for Social Multimedia at MediaEval 2013: Challenges, Data set, and Evaluation

Babak Loni, Martha A. Larson, Alessandro Bozzon, Luke Gottlieb
2013 MediaEval Benchmarking Initiative for Multimedia Evaluation  
This paper provides an overview of the Crowdsourcing for Multimedia Task at MediaEval 2013 multimedia benchmarking initiative.  ...  The 'basic human labels' were collected such that their noise levels would be characteristic of data gathered from crowdsourcing workers without using highly sophisticated quality control.  ...  The use of only a basic mechanism made it possible for participants of the task to explore more advanced quality control mechanisms in their approaches to the task.  ... 
dblp:conf/mediaeval/LoniLBG13 fatcat:6xjl24vd2ngxnde7el4oe24hfu

Sequence Effects in Crowdsourced Annotations

Nitika Mathur, Timothy Baldwin, Trevor Cohn
2017 Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing  
During rating tasks, seeing a low quality item may affect the score given to the next item either positively or negatively.  ...  In this paper, we explore sequence effects where annotations of an item are affected by the preceding items.  ...  Acknowledgments We thank the anonymous reviewers for their valuable feedback. This work was supported in part by the Australian Research Council.  ... 
doi:10.18653/v1/d17-1306 dblp:conf/emnlp/MathurBC17 fatcat:2oqafxinhzd55m54dstfxkufva

Brief survey of crowdsourcing for data mining

Guo Xintong, Wang Hongzhi, Yangqiu Song, Gao Hong
2014 Expert systems with applications  
Then we highlight several exemplar works in each component of the framework, including question designing, data mining and quality control.  ...  We first review the challenges and opportunities of data mining tasks using crowdsourcing, and summarize the framework of them.  ...  Quality Control: Due to the nature of crowdsourcing task, a quality control step is necessary for the result after the mining step.  ... 
doi:10.1016/j.eswa.2014.06.044 fatcat:ralo3tmglnat5jncasbz7jnye4

Pattern-Based Specification of Crowdsourcing Applications [chapter]

Alessandro Bozzon, Marco Brambilla, Stefano Ceri, Andrea Mauri, Riccardo Volonterio
2014 Lecture Notes in Computer Science  
We then discuss how workflows and patterns are managed by Crowd-Searcher, a system for designing, deploying and monitoring applications on top of crowd-based systems, including social networks and crowdsourcing  ...  Tasks interactions give rise to arbitrarily complex workflows. In this paper we propose methods and tools for designing crowdbased workflows as interacting tasks.  ...  the uncertainly ordered items.  ... 
doi:10.1007/978-3-319-08245-5_13 fatcat:jzwjfviyjbhzzjggfgjkqtai3i

Reactive crowdsourcing

Alessandro Bozzon, Marco Brambilla, Stefano Ceri, Andrea Mauri
2013 Proceedings of the 22nd international conference on World Wide Web - WWW '13  
An essential aspect for building effective crowdsourcing computations is the ability of "controlling the crowd", i.e. of dynamically adapting the behaviour of the crowdsourcing systems as response to the  ...  quantity and quality of completed tasks or to the availability and reliability of performers.  ...  assigns each item to one or more classes set of predefined classes Set of classes for each item Order The performer reorders the (top n) items in the input list n = number of top elements to be ordered  ... 
doi:10.1145/2488388.2488403 dblp:conf/www/BozzonBCM13 fatcat:xzvtm7xgn5gd7n5xmvbxisehrm

Crowd Database Operators [chapter]

Beth Trushkowsky
2016 Encyclopedia of Database Systems  
The use of crowdsourcing enables these systems to perform query operations that are well suited for people to compute, such as subjective comparisons, fuzzy matching for predicates and joins, entity resolution  ...  the cost of paying workers and the quality of the operator's output.  ...  The crowd completes comparison tasks that each contains s items; multiple worker responses per task are typically gathered for quality control purposes.  ... 
doi:10.1007/978-1-4899-7993-3_80660-1 fatcat:ztjfybgutrfhnj3sjwk73ngt2i

Designing Complex Crowdsourcing Applications Covering Multiple Platforms and Tasks

Alessandro Bozzon, Marco Brambilla, Stefano Ceri, Andrea Mauri, Riccardo Volonterio
2015 Journal of Web Engineering  
In this paper we propose methods and tools for designing crowd-based workflows as interacting tasks.  ...  We then discuss how workflows and patterns are managed by CrowdSearcher, a system for designing, deploying and monitoring applications on top of crowd-based systems, including social networks and crowdsourcing  ...  ordered items.  ... 
dblp:journals/jwe/BozzonBC0V15 fatcat:lx2wp72hbrge5mszggeao2u4y4

Information Extraction Meets Crowdsourcing: A Promising Couple

Christoph Lofi, Joachim Selke, Wolf-Tilo Balke
2012 Datenbank-Spektrum  
In these cases, another aspiring recent trend can be exploited to achieve the required extraction quality: explicit crowdsourcing of human intelligence tasks.  ...  Furthermore, we argue that for harnessing the full potential of either approach, true hybrid techniques must be considered.  ...  Therefore, alternative considerations for controlling quality are urgently needed for this scenario.  ... 
doi:10.1007/s13222-012-0092-8 fatcat:kmmhltrzijcoboqnxizaia7dtm

Using Visibility to Control Collective Attention in Crowdsourcing

Kristina Lerman, Tad Hogg
2013 AAAI Conference on Human Computation & Crowdsourcing  
Online crowdsourcing provides new opportunities for ordinary people to create original content.  ...  Content providers have innovative methods to identify high quality content based on crowdsourcing or peer recommendation.  ...  The random interface serves as a control by averaging over the effect of ordering, so the number of recommendations an item receives is related to its quality (Salganik, Dodds, and Watts 2006) .  ... 
dblp:conf/hcomp/LermanH13a fatcat:jl7iw3d5nff3llqgxfmwfket7i
« Previous Showing results 1 — 15 out of 8,542 results