Filters








1,750 Hits in 3.4 sec

Quality Control for Crowdsourced Enumeration
列挙型クラウドソーシングタスクのための品質管理法

Shunsuke Kajimura, Yukino Baba, Hiroshi Kajino, Hisashi Kashima
2016 Transactions of the Japanese society for artificial intelligence  
In this paper, we address a quality control problem of enumeration tasks, in which workers are asked to enumerate as many answers satisfying certain conditions as possible.  ...  Therefore, quality control is one of the central issues in crowdsourcing research.  ... 
doi:10.1527/tjsai.k-f79 fatcat:2iiv6vj3hvhbnnm7vm2vtrdtsa

Crowd-Powered Data Mining [article]

Chengliang Chai, Ju Fan, Guoliang Li, Jiannan Wang, Yudian Zheng
2018 arXiv   pre-print
We first give an overview of crowdsourcing, and then summarize the fundamental techniques, including quality control, cost control, and latency control, which must be considered in crowdsourced data mining  ...  Crowdsourcing is an effective way to harness the human cognitive ability to process these machine-hard tasks.  ...  Thus three core techniques must be considered in crowdsourcing: "quality control", "cost control", and "latency control".  ... 
arXiv:1806.04968v2 fatcat:b57ht5gmzzhd7dimwsyenzp4yq

Crowdsourced Data Management

Guoliang Li, Yudian Zheng, Ju Fan, Jiannan Wang, Reynold Cheng
2017 Proceedings of the 2017 ACM International Conference on Management of Data - SIGMOD '17  
We first give an overview of crowdsourcing, and then summarize the fundamental techniques, including quality control, cost control, and latency control, which must be considered in crowdsourced data management  ...  Crowdsourcing is an effective way to harness human cognitive abilities to process these computer-hard tasks, such as entity resolution, sentiment analysis, and image recognition.  ...  It is more challenging to control quality, latency and cost for mobile platforms.  ... 
doi:10.1145/3035918.3054776 dblp:conf/sigmod/LiZFWC17 fatcat:fsdmpcivdna4vnz27azhomgmwu

Crowdsourced Data Management: A Survey

Guoliang Li, Jiannan Wang, Yudian Zheng, Michael J. Franklin
2016 IEEE Transactions on Knowledge and Data Engineering  
We identify three important problems in crowdsourced data management. (1) Quality Control: Workers may return noisy or incorrect results so effective techniques are required to achieve high quality; (2  ...  ., the crowd) to apply human computation for such tasks. Thus, crowdsourced data management has become an area of increasing interest in research and industry.  ...  Quality Control The requester can select the quality-control techniques provided by the crowdsourcing platform, or design her/his own quality-control methods.  ... 
doi:10.1109/tkde.2016.2535242 fatcat:sit3comyvra4djqkl4xre3kj24

CDB

Guoliang Li, Haitao Yuan, Chengliang Chai, Ju Fan, Xueping Weng, Jian Li, Yudian Zheng, Yuanbing Li, Xiang Yu, Xiaohang Zhang
2017 Proceedings of the 2017 ACM International Conference on Management of Data - SIGMOD '17  
In fact, there are three optimization goals (i.e., smaller monetary cost, lower latency, and higher quality) in crowdsourcing, and it calls for a system to enable multi-goal optimization.  ...  We have also created a benchmark for evaluating crowd-powered databases.  ...  To improve crowdsourcing quality, a lot of quality control strategies are proposed.  ... 
doi:10.1145/3035918.3064036 dblp:conf/sigmod/LiCFWLZLYZY17 fatcat:f42oz3o2nrcn7nb3ctf6nwjl3i

What the Crowd Yields: Considerations when Crowdsourcing

Jeff Scagnelli
2013 Survey Practice  
The task replicated the process for store enumeration, sending respondents to known cosmetic stores within the geography based on Nielsen data.  ...  Crowdsourcing allows you to quickly reach a wide array of potential respondents and there is a need to ensure that you include these quality controls to reduce data quality issues.  ... 
doi:10.29115/sp-2013-0018 fatcat:wu6kjrkscbfd5pznmvnewjj7yu

Crowd Database Operators [chapter]

Beth Trushkowsky
2016 Encyclopedia of Database Systems  
The use of crowdsourcing enables these systems to perform query operations that are well suited for people to compute, such as subjective comparisons, fuzzy matching for predicates and joins, entity resolution  ...  the cost of paying workers and the quality of the operator's output.  ...  The crowd completes comparison tasks that each contains s items; multiple worker responses per task are typically gathered for quality control purposes.  ... 
doi:10.1007/978-1-4899-7993-3_80660-1 fatcat:ztjfybgutrfhnj3sjwk73ngt2i

Human-in-the-loop data integration

Guoliang Li
2017 Proceedings of the VLDB Endowment  
We first select some "beneficial" tasks to ask the crowd and then use transitivity and partial order to infer the answers of unasked tasks based on the crowdsourcing results of the asked tasks.  ...  We develop a crowd-powered database system CDB and deploy it on real crowdsourcing platforms. CDB allows users to utilize a SQL-like language for processing crowd-based queries.  ...  Improving the quality. Due to the openness of crowdsourcing, the crowd workers may yield low-quality or even noisy answers. Thus it is important to control the quality in crowdsourcing.  ... 
doi:10.14778/3137765.3137833 fatcat:xbgfksxhtravdd5g3pze3cxoau

Gamification in Crowdsourcing: A Review

Benedikt Morschheuser, Juho Hamari, Jonna Koivisto
2016 2016 49th Hawaii International Conference on System Sciences (HICSS)  
Crowdsourcing initiatives that provide more monotonous tasks most commonly used mere points and other simpler gamification implementations, whereas CS initiatives that seek for diverse and creative contributions  ...  Overall, the results of the review indicate that gamification has been an effective approach for increasing crowdsourcing participation.  ...  Homogenous non-emergent tasks are easily enumerable. Therefore, most crowdprocessing approaches reward the quantitative number of fulfilled tasks.  ... 
doi:10.1109/hicss.2016.543 dblp:conf/hicss/MorschheuserHK16 fatcat:widtetcswzhldgxwlkij72i3pe

Weaving Risk Identification into Crowdsourcing Lifecycle

Faouzi Kamoun, Dima Alhadidi, Zakaria Maamar
2015 Procedia Computer Science  
The proposed RBS calls for the need to pay close attention to risk monitoring during each phase of the crowdsourcing lifecycle.  ...  It then identifies the risk factors associated with each phase of the crowdsourcing lifecycle and discusses the impact of these identified risk factors on the crowdsourcing company.  ...  The design and implementation of internal quality control mechanisms can help in validating the data generated by the crowd.  ... 
doi:10.1016/j.procs.2015.07.181 fatcat:fouu6eesebhulc7yc67xn5mzaa

Crowdsourced enumeration queries

B. Trushkowsky, T. Kraska, M. J. Franklin, P. Sarkar
2013 2013 IEEE 29th International Conference on Data Engineering (ICDE)  
Hybrid human/computer database systems promise to greatly expand the usefulness of query processing by incorporating the crowd for data gathering and other tasks.  ...  These tools can also help drive query execution and crowdsourcing strategies. We evaluate our techniques using experiments on a popular crowdsourcing platform.  ...  In contrast, techniques have been proposed for quality control for individual set elements [5] , [8] .  ... 
doi:10.1109/icde.2013.6544865 dblp:conf/icde/TrushkowskyKFS13 fatcat:x2hyka745zamzftmnwcmvdcjmy

Influencing and Measuring Behaviour in Crowdsourced Activities [chapter]

Sandy J. J. Gould, Anna L. Cox, Duncan P. Brumby
2018 Human–Computer Interaction Series  
From this literature, we synthesize a set of design and implementation guidelines for crowdsourcing studies.  ...  The goal of this chapter it to clearly enumerate the difficulties of crowdsourcing psychometric data and to explore how, with careful planning and execution, these limitations can be overcome.  ...  This is important for requesters, as retaining high quality workers who are familiar with a task improves the quality of results returned.  ... 
doi:10.1007/978-3-319-73374-6_7 dblp:series/hci/GouldCB18 fatcat:dwsqao5bkna27moeerwjg5ellq

On Benchmarking for Crowdsourcing and Future of Work Platforms

Ria Mae Borromeo, Lei Chen, Abhishek Dubey, Sudeepa Roy, Saravanan Thirumuruganathan
2019 Zenodo  
The crowdsourcing community has done yeoman's work in designing effective algorithms for various key components, such as incentive design, task assignment and quality control.  ...  Online crowdsourcing platforms have proliferated over the last few years and cover a number of important domains, these platforms include from worker-task platforms such Amazon Mechanical Turk, worker-for  ...  The diversity of the crowd is often a better indicator of the quality of worker pool especially for knowledge intensive crowdsourcing tasks.  ... 
doi:10.5281/zenodo.6793148 fatcat:72mhfxfobfb2hobqm54hjq3ztq

On Benchmarking for Crowdsourcing and Future of Work Platforms

Ria Mae Borromeo, Lei Chen, Abhishek Dubey, Sudeepa Roy, Saravanan Thirumuruganathan
2019 IEEE Data Engineering Bulletin  
The crowdsourcing community has done yeoman's work in designing effective algorithms for various key components, such as incentive design, task assignment and quality control.  ...  We believe that developing appropriate benchmarks for crowdsourcing will ignite further innovations.  ...  The diversity of the crowd is often a better indicator of the quality of worker pool especially for knowledge intensive crowdsourcing tasks.  ... 
dblp:journals/debu/Borromeo0DRT19 fatcat:umznjewa2zhjnpcm5rbgsdtbhy

Crowdsourced Data Management: Industry and Academic Perspectives

Adam Marcus, Aditya Parameswaran
2015 Foundations and Trends in Databases  
Crowdsourcing and human computation enable organizations to accomplish tasks that are currently not possible for fully automated techniques to complete, or require more flexibility and scalability than  ...  work quality, latency, or cost.  ...  Quality Control The quality control mechanisms of each platform relate to and depend on the types of tasks the workers are doing on that platform.  ... 
doi:10.1561/1900000044 fatcat:rva7dinutnbnlj2hvm4j2hmhge
« Previous Showing results 1 — 15 out of 1,750 results