1,814 Hits in 4.1 sec

Fault-Tolerant Entity Resolution with the Crowd [article]

Anja Gruenheid and Besmira Nushi and Tim Kraska and Wolfgang Gatterbauer and Donald Kossmann
2015 arXiv   pre-print
Furthermore, we examine task-to-worker assignment strategies as well as task parallelization techniques in terms of their cost and quality trade-offs in this paper.  ...  Based on both synthetic and crowdsourced datasets, we then draw conclusions on how to minimize cost while maintaining high quality ER results.  ...  Entity resolution strategies that are based on consensus decision usually allocate a fixed repetition budget for each crowdsourcing task.  ... 
arXiv:1512.00537v1 fatcat:cfkaaju3kndnbezglerqvyfs54

Deep Learning for Mobile Crowdsourcing Techniques, Methods, and Challenges: A Survey

Bingchen Liu, Weiyi Zhong, Jushi Xie, Lingzhen Kong, Yihong Yang, Chuang Lin, Hao Wang
2021 Mobile Information Systems  
In view of this, we review the current research status of deep learning for mobile crowdsourcing from the perspectives of techniques, methods, and challenges.  ...  resolution for solving complex computational tasks with quick response requirements.  ...  A requester can assign a broad range of complex tasks which are time consuming for workers, and workers only with a mobile device can participate in the task. In Ref.  ... 
doi:10.1155/2021/6673094 doaj:526bd22b22354287b58ac08179088be1 fatcat:gotbqx6tmvglxmciyhmxezlvfq

Improving Human-Labeled Data through Dynamic Automatic Conflict Resolution [article]

David Q. Sun, Hadas Kotek, Christopher Klein, Mayank Gupta, William Li, Jason D. Williams
2020 arXiv   pre-print
This paper develops and implements a scalable methodology for (a) estimating the noisiness of labels produced by a typical crowdsourcing semantic annotation task, and (b) reducing the resulting error of  ...  This makes DACR not only more accurate but also available to a broad range of labeling tasks.  ...  Subsequently, some post-processing technique is used to create a consensus label that is recorded as the ground truth.  ... 
arXiv:2012.04169v1 fatcat:kxpbdrwdxnhe3c4gwkxbrg5tkm

Learning an accurate entity resolution model from crowdsourced labels

Jingjing Wang, Satoshi Oyama, Masahito Kurihara, Hisashi Kashima
2014 Proceedings of the 8th International Conference on Ubiquitous Information Management and Communication - ICUIMC '14  
First, we evaluated the quality of crowd-generated labels for actual entity resolution data sets.  ...  method that combines multiple Laplacians directly by using crowdsourced data.  ...  There are a number of papers on designing fundamental algorithms for crowdsourcing [9; 14; 15] and a number of papers on developing systems for entity resolution in Web search result [13; 1; 2; 21;  ... 
doi:10.1145/2557977.2558060 dblp:conf/icuimc/WangOKK14 fatcat:2hwx52fnfrcqlbge7dldtx767e

Quality Control in Crowdsourcing Systems: Issues and Directions

M. Allahbakhsh, B. Benatallah, A. Ignjatovic, H. R. Motahari-Nezhad, E. Bertino, S. Dustdar
2013 IEEE Internet Computing  
This article proposes a framework for characterizing various dimensions of quality control in crowdsourcing systems, a critical issue.  ...  As a new distributed computing model, crowdsourcing lets people leverage the crowd's intelligence and wisdom toward solving problems.  ...  We propose a taxonomy for quality in crowdsourcing systems, as Figure 1 illustrates.  ... 
doi:10.1109/mic.2013.20 fatcat:p3xpvckwcrfplcmoh5xq4e26pm

Crowdsourcing Human Annotation on Web Page Structure

Shuguang Han, Peng Dai, Praveen Paritosh, David Huynh
2016 ACM Transactions on Intelligent Systems and Technology  
In this article, we present a new crowdsourcing system, called Wernicke, to provide annotations for web information extraction.  ...  In doing so, we discover several things: (1) Many behavioral features are significant predictors for crowdsourcing quality. (2) The behavioral-feature-based method outperforms performance-based methods  ...  Crowdsourcing systems enable task owners to recruit many nonexpert workers.  ... 
doi:10.1145/2870649 fatcat:bhqlnn5pdre7pbwbevspfeljcq

Exposing ambiguities in a relation-extraction gold standard with crowdsourcing [article]

Tong Shu Li, Benjamin M. Good, Andrew I. Su
2015 arXiv   pre-print
This work rein-forces the power of crowdsourcing in the process of assembling gold standards for relation extraction.  ...  We implemented and evaluated a microtask crowdsourcing approach that can produce a gold standard for extracting drug-disease relations.  ...  We would also like to thank Alex Bravo Serrano and Laura Furlong from the EU-ADR for sharing the original expert annotation data.  ... 
arXiv:1505.06256v1 fatcat:d2dtunzzmbb6xgrevskrctpplu

Integrating camera imagery, crowdsourcing, and deep learning to improve high-frequency automated monitoring of snow at continental-to-global scales

Margaret Kosmala, Koen Hufkens, Andrew D. Richardson, Seyedali Mirjalili
2018 PLoS ONE  
It holds broad potential for real-time large-scale acquisition and processing of ecological and environmental data in support of monitoring, management, and research objectives.  ...  Snow is important for local to global climate and surface hydrology, but spatial and temporal heterogeneity in the extent of snow cover make accurate, fine-scale mapping and monitoring of snow an enormous  ...  We appreciate the work of Don Aubrecht, Steve Klosterman, Morgan Furze, Miriam Johnston, who provided expert Knowxel classifications, and to Tom Milliman for PhenoCam data management.  ... 
doi:10.1371/journal.pone.0209649 pmid:30589858 pmcid:PMC6307743 fatcat:pttx56mphravxmcpgw2sy2xzne

Use of Mechanical Turk as a MapReduce Framework for Macular OCT Segmentation

Aaron Y. Lee, Cecilia S. Lee, Pearse A. Keane, Adnan Tufail
2016 Journal of Ophthalmology  
Each HIT was completed in an average of 4.43 minutes.Conclusions.Amazon Mechanical Turk provides a cost-effective, scalable, high-availability infrastructure for manual segmentation of OCT images.  ...  Each Human Intelligence Task was set to$0.01 and required the user to draw five lines to outline the sublayers of the retinal OCT image after being shown example images.  ...  In this system, simple Human Intelligence Tasks (HITs) are submitted to online untrained users for a small compensation.  ... 
doi:10.1155/2016/6571547 pmid:27293877 pmcid:PMC4879255 fatcat:al3a7jybsjezxcmnmtw27aq2ae

Crowdsourced Data Management: Industry and Academic Perspectives

Adam Marcus, Aditya Parameswaran
2015 Foundations and Trends in Databases  
Crowdsourcing and human computation enable organizations to accomplish tasks that are currently not possible for fully automated techniques to complete, or require more flexibility and scalability than  ...  Several academic researchers from diverse areas ranging from the social sciences to computer science have embraced crowdsourcing as a research area, resulting in algorithms and systems that improve crowd  ...  One participant explained that algorithms aren't as effective on qualitative tasks. They expanded that "Most answers are richly qualitative or based on consensus.  ... 
doi:10.1561/1900000044 fatcat:rva7dinutnbnlj2hvm4j2hmhge


Anhong Guo, Xiang 'Anthony' Chen, Jeffrey P. Bigham
2015 Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA '15  
To address this problem, we present ApplianceReader -a system that combines a wearable point-of-view camera with on-demand crowdsourcing and computer vision to make appliance interfaces accessible.  ...  Computer vision techniques then track the user's finger pointing at the controls and read out the labels previously provided by the crowd.  ...  Conclusion We are developing ApplianceReader -a wearable, crowdsourced, vision-based system to make appliances accessible for visually impaired users.  ... 
doi:10.1145/2702613.2732755 dblp:conf/chi/GuoCB15 fatcat:5yl7dew2azhvpim6igahs44jti

Evaluating Design Solutions Using Crowds

Jin Bao, Yasuaki Sakamoto, Jeffrey V. Nickerson
2011 Social Science Research Network  
The results have pragmatic implications, suggesting that evaluation methods should be assigned in relation to the distribution of quality present at each stage of crowdsourcing.  ...  The crowd prefers participating in prediction voting.  ...  The rating task focused evaluators on the entire range of solutions. Thus, this technique was not good for discovering the very best out of the good solutions.  ... 
doi:10.2139/ssrn.2201651 fatcat:wlzwhig63jfa5ebkmenwzuxcli

Crowd-Enhanced Cloud Services: Issues and Directions

Saeed Arbabi, Mohammad Allah Bakhsh, Mohsen Sharifi
2015 International Journal of Computer Applications  
Crowdsourcing in combination with the machine automated techniques.  ...  There are large numbers of cases in which Cloud techniques solely are not able to solve the job due to the nature of the tasks.  ...  Crowdsourcing systems enlist a multitude of humans to help solve a wide variety of problems [3] ranging from labeling images [4] , to finding 3-D protein structures [5] , to classifying galaxies in  ... 
doi:10.5120/20676-3486 fatcat:lqejfkgvk5attgrn5i5rjp3oni

Optimizing Open-Ended Crowdsourcing: The Next Frontier in Crowdsourced Data Management [article]

Aditya Parameswaran, Akash Das Sarma, Vipul Venkataraman
2016 arXiv   pre-print
frontier in crowdsourced data management.  ...  We describe the approaches that we've found to be effective for open-ended crowdsourcing, drawing from our experiences in this space.  ...  Area 2: Crowdsourcing Systems and Toolkits.  ... 
arXiv:1610.05377v1 fatcat:5h33kbzrn5ffdlmjvyfodfo5j4

Expert Sourcing to Support the Identification of Model Elements in System Descriptions [chapter]

Marta Sabou, Dietmar Winkler, Sanja Petrovic
2017 Lecture Notes in Business Information Processing  
Based on these lessons learned we propose a new task design for expert sourcing EMEs.  ...  For example, given a text-based system description and a corresponding model such as an Extended Entity Relationship (EER) diagram, experts are guided towards inspecting the model based on socalled expected  ...  We would like to thank the participants of the software quality course at Vienna University of Technology in the winter term 2016/2017 for participating in the study.  ... 
doi:10.1007/978-3-319-71440-0_5 fatcat:u7hn5fdvljhnfkrl3ualufpxl4
« Previous Showing results 1 — 15 out of 1,814 results