The K-armed dueling bandits problem

Yisong Yue, Josef Broder, Robert Kleinberg, Thorsten Joachims
2012 Journal of computer and system sciences (Print)  
We study a partial-information online-learning problem where actions are restricted to noisy comparisons between pairs of strategies (also known as bandits). In contrast to conventional approaches that require the absolute reward of the chosen strategy to be quantifiable and observable, our setting assumes only that (noisy) binary feedback about the relative reward of two chosen strategies is available. This type of relative feedback is particularly appropriate in applications where absolute
more » ... ards have no natural scale or are difficult to measure (e.g., user-perceived quality of a set of retrieval results, taste of food, product attractiveness), but where pairwise comparisons are easy to make. We propose a novel regret formulation in this setting, as well as present an algorithm that achieves information-theoretically optimal regret bounds (up to a constant factor).
doi:10.1016/j.jcss.2011.12.028 fatcat:js6gr4zh6vamfcpu3o4ljl2bqy