A method for evaluating discoverability and navigability of recommendation algorithms

Daniel Lamprecht, Markus Strohmaier, Denis Helic
2017 Computational Social Networks  
Websites with large collections of items need to support three ways of information retrieval: (1) retrieval of familiar items; (2) retrieval of items that cannot be explicitly described, but will be recognized once retrieved; and (3) serendipitous discovery [1] . For a website with a large collection of items, such as an e-commerce website or a video platform, (1) can be enabled with a full-text search function. For (2) and (3), however, a search function is generally not sufficient. These
more » ... of information retrieval are, therefore, often supported by recommendations that connect items and enable discovery and navigation. Users have been found to enjoy perusing item collections such as e-commerce sites or recommender systems without the immediate intention of making a purchase [2] . Flickr users predominately discover new images via social browsing [3] . More generally, some users prefer navigation to direct search even when they know the target [4] . In exploratory scenarios, the knowledge gained along the way provides context and aids in learning Abstract Recommendations are increasingly used to support and enable discovery, browsing, and exploration of items. This is especially true for entertainment platforms such as Netflix or YouTube, where frequently, no clear categorization of items exists. Yet, the suitability of a recommendation algorithm to support these use cases cannot be comprehensively evaluated by any recommendation evaluation measures proposed so far. In this paper, we propose a method to expand the repertoire of existing recommendation evaluation techniques with a method to evaluate the discoverability and navigability of recommendation algorithms. The proposed method tackles this by means of first evaluating the discoverability of recommendation algorithms by investigating structural properties of the resulting recommender systems in terms of bow tie structure, and path lengths. Second, the method evaluates navigability by simulating three different models of information seeking scenarios and measuring the success rates. We show the feasibility of our method by applying it to four non-personalized recommendation algorithms on three data sets and also illustrate its applicability to personalized algorithms. Our work expands the arsenal of evaluation techniques for recommendation algorithms, extends from a one-click-based evaluation towards multi-click analysis, and presents a general, comprehensive method to evaluating navigability of arbitrary recommendation algorithms.
doi:10.1186/s40649-017-0045-3 pmid:29266112 pmcid:PMC5732611 fatcat:x3nbulcksnggbjencyr2b6zv2e