Cheaper and Better: Selecting Good Workers for Crowdsourcing [article]

Hongwei Li, Qiang Liu
2015 arXiv   pre-print
Crowdsourcing provides a popular paradigm for data collection at scale. We study the problem of selecting subsets of workers from a given worker pool to maximize the accuracy under a budget constraint. One natural question is whether we should hire as many workers as the budget allows, or restrict on a small number of top-quality workers. By theoretically analyzing the error rate of a typical setting in crowdsourcing, we frame the worker selection problem into a combinatorial optimization
more » ... m and propose an algorithm to solve it efficiently. Empirical results on both simulated and real-world datasets show that our algorithm is able to select a small number of high-quality workers, and performs as good as, sometimes even better than, the much larger crowds as the budget allows.
arXiv:1502.00725v1 fatcat:z56gphuqxvfeblp6lwfwihwm4y