Counterfactual Evaluation and Learning for Search, Recommendation and Ad Placement

Thorsten Joachims, Adith Swaminathan
<span title="">2016</span> <i title="ACM Press"> <a target="_blank" rel="noopener" href="" style="color: black;">Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval - SIGIR &#39;16</a> </i> &nbsp;
Online metrics measured through A/B tests have become the gold standard for many evaluation questions. But can we get the same results as A/B tests without actually fielding a new system? And can we train systems to optimize online metrics without subjecting users to an online learning algorithm? This tutorial summarizes and unifies the emerging body of methods on counterfactual evaluation and learning. These counterfactual techniques provide a well-founded way to evaluate and optimize online
more &raquo; ... trics by exploiting logs of past user interactions. In particular, the tutorial unifies the causal inference, information retrieval, and machine learning view of this problem, providing the basis for future research in this emerging area of great potential impact. Supplementary material and resources are available online at
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="">doi:10.1145/2911451.2914803</a> <a target="_blank" rel="external noopener" href="">dblp:conf/sigir/JoachimsS16</a> <a target="_blank" rel="external noopener" href="">fatcat:nfbcecjnhvffblezblfvqqzxu4</a> </span>
<a target="_blank" rel="noopener" href="" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href=""> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> </button> </a>