Image-Based Recommendations on Styles and Substitutes

Julian McAuley, Christopher Targett, Qinfeng Shi, Anton van den Hengel
2015 Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval - SIGIR '15  
Humans inevitably develop a sense of the relationships between objects, some of which are based on their appearance. Some pairs of objects might be seen as being alternatives to each other (such as two pairs of jeans), while others may be seen as being complementary (such as a pair of jeans and a matching shirt). This information guides many of the choices that people make, from buying clothes to their interactions with each other. We seek here to model this human sense of the relationships
more » ... een objects based on their appearance. Our approach is not based on fine-grained modeling of user annotations but rather on capturing the largest dataset possible and developing a scalable method for uncovering human notions of the visual relationships within. We cast this as a network inference problem defined on graphs of related images, and provide a large-scale dataset for the training and evaluation of the same. The system we develop is capable of recommending which clothes and accessories will go well together (and which will not), amongst a host of other applications. Books WNN 66.5% 62.8% 63.3% 65.4% K = 10 70.1% 68.6% 69.3% 68.1% K = 100 71.2% 69.8% 71.2% 68.6% Cell Phones and Accessories WNN 73.4% 66.4% 69.1% 79.3% K = 10 84.3% 78.9% 78.7% 83.1% K = 100 85.9% 83.1% 83.2% 87.7% Clothing, Shoes, and Jewelry WNN · 77.2% 74.2% 78.3% K = 10 · 87.5% 84.7% 89.7% K = 100 · 88.8% 88.7% 92.5% Digital Music WNN 60.2% 56.7% 62.2% 53.3% K = 10 68.7% 60.9% 74.7% 56.0% K = 100 72.3% 63.8% 76.2% 59.0% Electronics WNN 76.5% 73.8% 67.6% 73.5% K = 10 83.6% 80.3% 77.8% 79.6% K = 100 86.4% 84.0% 82.6% 83.2% Grocery and Gourmet Food WNN · 69.2% 70.7% 68.5% K = 10 · 77.8% 81.2% 79.6% K = 100 · 82.5% 85.2% 84.5% Home and Kitchen WNN 75.1% 68.3% 70.4% 76.6% K = 10 78.5% 80.5% 78.8% 79.3% K = 100 81.6% 83.8% 83.4% 83.2% Movies and TV WNN 66.8% 65.6% 61.6% 59.6% K = 10 71.9% 69.6% 72.8% 67.6% K = 100 72.3% 70.0% 77.3% 70.7% Musical Instruments
doi:10.1145/2766462.2767755 dblp:conf/sigir/McAuleyTSH15 fatcat:z5xegwekdvhm5au6my6i64lfzq