Filters








82 Hits in 5.2 sec

Optimistic Knowledge Gradient Policy for Optimal Budget Allocation in Crowdsourcing

Xi Chen, Qihang Lin, Dengyong Zhou
2013 International Conference on Machine Learning  
To solve the computational challenge, we propose a novel approximate policy which is called optimistic knowledge gradient.  ...  Using the dynamic programming (DP) algorithm, one can obtain the optimal allocation policy for a given budget. However, DP is computationally intractable.  ...  Acknowledgements The authors would like to thank Qiang Liu for sharing the code of KOS and BP; and Jing Xie and Peter Frazier for sharing their code for infinite-horizon Gittins index.  ... 
dblp:conf/icml/ChenLZ13 fatcat:dnzvsrhi7vcw7chxoz7s6mxelq

Statistical Decision Making for Optimal Budget Allocation in Crowd Labeling

Xi Chen, Qihang Lin, Dengyong Zhou
2014 Social Science Research Network  
Optimistic Knowledge Gradient Policy for Optimal Budget Allocation in Crowdsourcing. In Proceedings of Inter- national Conference on Machine Learning (ICML), 2013 • Xi Chen, Paul N.  ...  In Proceedings of International Conference on Machine Learning (ICML), 2014 • Chong Wang, Xi Chen, Alex Smola, and Eric Xing. Variance Reduction for Stochastic Gradient Optimization.  ... 
doi:10.2139/ssrn.2408163 fatcat:6i33zzawmvcnxof4mvcxkwt7uq

Dynamic Task Allocation for Crowdsourcing Settings [article]

Angela Zhou, Irineo Cabreros, Karan Singh
2017 arXiv   pre-print
We consider the problem of optimal budget allocation for crowdsourcing problems, allocating users to tasks to maximize our final confidence in the crowdsourced answers.  ...  Such an optimized worker assignment method allows us to boost the efficacy of any popular crowdsourcing estimation algorithm.  ...  Another approach models the assignment problem as a Markov Decision Process and derives the Optimistic Knowledge Gradient, which computes the conditional expectation of choosing certain workers, assuming  ... 
arXiv:1701.08795v2 fatcat:3lvt3e2uhfdjznkhgqkgud4oty

Learning Large Logic Programs By Going Beyond Entailment

Andrew Cropper, Sebastijan Dumančic
2020 Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence  
We implement our idea in Brute, a new ILP system which uses best-first search, guided by an example-dependent loss function, to incrementally build programs.  ...  A major challenge in inductive logic programming (ILP) is learning large programs. We argue that a key limitation of existing systems is that they use entailment to guide the hypothesis search.  ...  Conclusions In this paper, we propose an online policy for the budget allocation problem in crowdsourced clustering.  ... 
doi:10.24963/ijcai.2020/283 dblp:conf/ijcai/WangCLL20 fatcat:kpwu3w4zybhfnmup27245wplmi

Statistical Decision Making for Optimal Budget Allocation in Crowd Labeling [article]

Xi Chen, Qihang Lin, Dengyong Zhou
2014 arXiv   pre-print
To solve this challenge, we propose a computationally efficient approximate policy, called optimistic knowledge gradient policy.  ...  Since data instances have different levels of labeling difficulty and workers have different reliability, it is desirable to have an optimal policy to allocate the budget among all instance-worker pairs  ...  , we propose an efficient approximate policy, optimistic knowledge gradient; (3) the proposed MDP framework can be used as a general framework to address various budget allocation problems in crowdsourcing  ... 
arXiv:1403.3080v2 fatcat:tntusblngnegpoqiatzyk4nrwq

Bayes-Optimal Effort Allocation in Crowdsourcing: Bounds and Index Policies [article]

Weici Hu, Peter I. Frazier
2015 arXiv   pre-print
In an approach similar in spirit to the Whittle index for restless multiarmed bandits, we provide an index policy for effort allocation in crowdsourcing and demonstrate numerically that it outperforms  ...  We consider effort allocation in crowdsourcing, where we wish to assign labeling tasks to imperfect homogeneous crowd workers to maximize overall accuracy in a continuous-time Bayesian setting, subject  ...  Acknowledgment The authors were partially supported by NSF CAREER CMMI-1254298, NSF IIS-1247696, NSF CMMI-1536895, AFOSR FA9550-12-1-0200, AFOSR FA9550-15-1-0038, and the ACSF AVF (Atkinson Center for  ... 
arXiv:1512.09204v1 fatcat:tosaxx6k7rdnbaiifoj3zw2ikq

Testing independence with high-dimensional correlated samples

Xi Chen, Weidong Liu
2018 Annals of Statistics  
Optimistic Knowledge Gradient Policy for Optimal Budget Allocation in Crowdsourcing.  ...  Statistical Decision Making for Optimal Budget Allocation in Crowd Labeling. Journal of Machine Learning Research, 16: 1-46, 2015.  ...  Journal Reviewer 11/2017 Marquis Who's Who, Who's Who in the World for Achievements in Data Science, http://www.24-7pressrelease.com/press-release/ xi-chen-recognized-by-marquis-whos-who-for-achievements-in-data-science  ... 
doi:10.1214/17-aos1571 fatcat:poic4p6vdrer5iiqivvs6hjloe

Large-Scale Markov Decision Problems with KL Control Cost and its Application to Crowdsourcing

Yasin Abbasi-Yadkori, Peter L. Bartlett, Xi Chen, Alan Malek
2015 International Conference on Machine Learning  
We demonstrate the efficacy of our approach by optimizing a policy for budget allocation in crowd labeling, an important crowdsourcing application.  ...  Since the computational and statistical cost of finding the optimal policy scales with the size of the state space, we focus on searching for near-optimality in a low-dimensional family of policies.  ...  Therefore, we construct P 0 based on the optimistic knowledge gradient policy (Opt-KG) proposed in Chen et al. (2013) , which was shown to work quite well in practice.  ... 
dblp:conf/icml/Abbasi-YadkoriB15 fatcat:7fb6h7blifeepauhapd6k77omi

Bandit-Based Task Assignment for Heterogeneous Crowdsourcing [article]

Hao Zhang, Yao Ma, Masashi Sugiyama
2015 arXiv   pre-print
We consider a task assignment problem in crowdsourcing, which is aimed at collecting as many reliable labels as possible within a limited budget.  ...  In this paper, we propose a contextual bandit formulation for task assignment in heterogeneous crowdsourcing, which is able to deal with the exploration-exploitation trade-off in worker selection.  ...  Optimistic Knowledge Gradient Optimistic Knowledge Gradient (OptKG) (Chen et al., 2013) uses an N-coin-tossing model, formulates the task assignment problem as a Bayesian Markov decision process (MDP  ... 
arXiv:1507.05800v1 fatcat:teulumpzejctvfbjcatprufxbq

Optimal Stopping and Worker Selection in Crowdsourcing: an Adaptive Sequential Probability Ratio Test Framework

Xiaoou Li, Yunxiao Chen, Xi Chen, Jingchen Liu, Zhiliang Ying
2021 Statistica sinica  
under a certain budget constraint.  ...  Our motivating application comes from binary labeling tasks in crowdsourcing, where a requestor needs to simultaneously decide which worker to choose to provide a label and when to stop collecting labels  ...  Acknowledgment The authors thank the editors and two referees for their constructive comments.  ... 
doi:10.5705/ss.202018.0300 fatcat:t33sz6v7dzhe3iji6nyvl266ci

Efficient crowdsourcing of crowd-generated microtasks

Abigail Hotaling, James P. Bagrow, Haoran Xie
2020 PLoS ONE  
In this paper, we introduce cost forecasting as a means for a crowdsourcer to use efficient crowdsourcing algorithms with a growing set of microtasks.  ...  training data for machine learning applications and improving the performance of user-generated content and question-answering platforms.  ...  Acknowledgments We thank Paul Hines and Hamid Ossareh for helpful comments. Author Contributions Conceptualization: Abigail Hotaling, James P. Bagrow. Data curation: Abigail Hotaling.  ... 
doi:10.1371/journal.pone.0244245 pmid:33332455 fatcat:iebvwzm4bfacboxkwaku5nt34u

Bandit-Based Task Assignment for Heterogeneous Crowdsourcing

Hao Zhang, Yao Ma, Masashi Sugiyama
2015 Neural Computation  
We consider a task assignment problem in crowdsourcing, which is aimed at collecting as many reliable labels as possible within a limited budget.  ...  In this paper, we propose a contextual bandit formulation for task assignment in heterogeneous crowdsourcing, which is able to deal with the exploration-exploitation trade-off in worker selection.  ...  Optimistic Knowledge Gradient Optimistic Knowledge Gradient (OptKG) (Chen et al., 2013) uses an N -coin-tossing model, formulates the task assignment problem as a Bayesian Markov decision process (MDP  ... 
doi:10.1162/neco_a_00782 pmid:26378878 fatcat:tmriuekiqzbqbgmt6lfuuxgtom

Restless Bandits with Many Arms: Beating the Central Limit Theorem [article]

Xiangyu Zhang, Peter I. Frazier
2021 arXiv   pre-print
We study the growth of the optimality gap, i.e., the loss in expected performance compared to an optimal policy, for such policies in a classical asymptotic regime proposed by Whittle in which N grows  ...  Thus, there is substantial value in understanding the performance of index policies and other policies that can be computed efficiently for large N.  ...  The Online Knowledge Gradient and the Optimistic Knowledge Gradient, however, seem to have suboptimality that scales linearly with N . Figure 2 2 Crowdsourced labeling.  ... 
arXiv:2107.11911v1 fatcat:tinezne3ejhenlynjocg2ofb4q

Sequential crowdsourced labeling as an epsilon-greedy exploration in a Markov Decision Process

Vikas C. Raykar, Priyanka Agrawal
2014 International Conference on Artificial Intelligence and Statistics  
With the goal of reducing the labeling cost, we introduce the notion of sequential crowdsourced labeling, where instead of asking for all the labels in one shot we acquire labels from annotators sequentially  ...  Crowdsourcing marketplaces are widely used for curating large annotated datasets by collecting labels from multiple annotators.  ...  Recently [7] also cast the sequential labeling problem as an MDP and proposed to use the optimistic knowledge gradient as an approximate allocation policy.  ... 
dblp:conf/aistats/RaykarA14 fatcat:ei26cr442nemljnup7pob6i6ra

Cheaper and Better: Selecting Good Workers for Crowdsourcing [article]

Hongwei Li, Qiang Liu
2015 arXiv   pre-print
Crowdsourcing provides a popular paradigm for data collection at scale.  ...  By theoretically analyzing the error rate of a typical setting in crowdsourcing, we frame the worker selection problem into a combinatorial optimization problem and propose an algorithm to solve it efficiently  ...  Optimistic knowledge gradient policy for optimal budget allocation in crowdsourcing. In ICML, 2013. C. Ho, S. Jabbari, and J.W. Vaughan. Adaptive task assignment for crowdsourced classification.  ... 
arXiv:1502.00725v1 fatcat:z56gphuqxvfeblp6lwfwihwm4y
« Previous Showing results 1 — 15 out of 82 results