Dueling Bandits with Dependent Arms [article]

Bangrui Chen, Peter I. Frazier
<span title="2017-06-15">2017</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
We study dueling bandits with weak utility-based regret when preferences over arms have a total order and carry observable feature vectors. The order is assumed to be determined by these feature vectors, an unknown preference vector, and a known utility function. This structure introduces dependence between preferences for pairs of arms, and allows learning about the preference over one pair of arms from the preference over another pair of arms. We propose an algorithm for this setting called
more &raquo; ... mparing The Best (CTB), which we show has constant expected cumulative weak utility-based regret. We provide a Bayesian interpretation for CTB, an implementation appropriate for a small number of arms, and an alternate implementation for many arms that can be used when the input parameters satisfy a decomposability condition. We demonstrate through numerical experiments that CTB with appropriate input parameters outperforms all benchmarks considered.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1605.08838v2">arXiv:1605.08838v2</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/hkmcgyqzszh6fpokstlbpvx6kq">fatcat:hkmcgyqzszh6fpokstlbpvx6kq</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200929032744/https://arxiv.org/pdf/1605.08838v2.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/3c/ba/3cbacfecfd3b38096bc896e8f9d490a201d800c1.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1605.08838v2" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>