Filters








299 Hits in 4.9 sec

IR Evaluation and Learning in the Presence of Forbidden Documents

David Carmel, Nachshon Cohen, Amir Ingber, Elad Kravi
2022 Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval  
We propose a learning to rank and filter (LTRF) framework that is specifically designed to optimize nDCG f , by learning a ranking model and optimizing a filtering threshold used for discarding documents  ...  , and reliability of nDCG f for this task.  ...  Optimizing nDCG f The core element of LTR is the loss function to minimize while searching for a ranking model.  ... 
doi:10.1145/3477495.3532006 fatcat:xrm6aj44ane4tc5sifsrn2qjqq

On NDCG Consistency of Listwise Ranking Methods

Pradeep Ravikumar, Ambuj Tewari, Eunho Yang
2011 Journal of machine learning research  
We characterize NDCG consistency of surrogate losses to discover a surprising fact: several commonly used surrogates are NDCG inconsistent.  ...  State of the art listwise approaches replace NDCG with a surrogate loss that is easier to optimize.  ...  Inconsistency of common surrogates We have seen above that the optimal score vector (for minimizing NDCG loss) is not obtained simply from the sorted order of E [G(r)], but rather from the sorted order  ... 
dblp:journals/jmlr/RavikumarTY11 fatcat:fhctoit2xjazvmroddg2fjaz2y

Optimize What You Evaluate With: A Simple Yet Effective Framework For Direct Optimization Of IR Metrics [article]

Hai-Tao Yu
2020 arXiv   pre-print
The performance of learning-to-rank methods is commonly evaluated using rank-sensitive metrics, such as average precision (AP) and normalized discounted cumulative gain (nDCG).  ...  To validate the effectiveness of the proposed framework for direct optimization of IR metrics, we conduct a series of experiments on the widely used benchmark collection MSLRWEB30K.  ...  We name it as the problem of optimization inconsistency.  ... 
arXiv:2008.13373v1 fatcat:vwf3cka3dvaadapmfkkkmybkny

Robust sparse rank learning for non-smooth ranking measures

Zhengya Sun, Tao Qin, Qing Tao, Jue Wang
2009 Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval - SIGIR '09  
In this paper, we consider the sparse algorithms to directly optimize the Normalized Discounted Cumulative Gain (NDCG) which is a widely-used ranking measure.  ...  We begin by establishing a reduction framework under which we reduce ranking, as measured by NDCG, to the importance weighted pairwise classification.  ...  P ) (9) The ranker with maximum NDCG expected gain is described as the "Bayes optimal ranker" in this paper.  ... 
doi:10.1145/1571941.1571987 dblp:conf/sigir/SunQTW09 fatcat:nvfaso4ffjcqrnuyb5efyqxasi

Deep Semantic Ranking Based Hashing for Multi-Label Image Retrieval [article]

Fang Zhao, Yongzhen Huang, Liang Wang, Tieniu Tan
2015 arXiv   pre-print
An effective scheme based on surrogate loss is used to solve the intractable optimization problem of nonsmooth and multivariate ranking measures involved in the learning procedure.  ...  The complex multilevel semantic structure of images associated with multiple labels have not yet been well explored.  ...  By using a triplet representation for listwise supervision, ranking-based supervised hashing (RSH) [29] minimizes the inconsistency of ranking order between the hamming and original spaces to keep global  ... 
arXiv:1501.06272v2 fatcat:x6z6lomzqzhc3k7dcrwsrqhsey

Deep semantic ranking based hashing for multi-label image retrieval

Fang Zhao, Yongzhen Huang, Liang Wang, Tieniu Tan
2015 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)  
An effective scheme based on surrogate loss is used to solve the intractable optimization problem of nonsmooth and multivariate ranking measures involved in the learning procedure.  ...  The complex multilevel semantic structure of images associated with multiple labels have not yet been well explored.  ...  By using a triplet representation for listwise supervision, ranking-based supervised hashing (RSH) [29] minimizes the inconsistency of ranking order between the hamming and original spaces to keep global  ... 
doi:10.1109/cvpr.2015.7298763 dblp:conf/cvpr/ZhaoHWT15 fatcat:t2u7qqcqgzg7fpd5jg46tikh5i

Diagnostic Evaluation of Policy-Gradient-Based Ranking

Hai-Tao Yu, Degen Huang, Fuji Ren, Lishuang Li
2021 Electronics  
This paper is motivated by the observation that recent ranking methods based on either reinforcement learning or adversarial learning boil down to policy-gradient-based optimization.  ...  policy-gradient-based ranking methods are effective.  ...  This work was supported by JSPS KAKENHI, Grant Number 19H04215. Conflicts of Interest: The authors declare no conflict of interest.  ... 
doi:10.3390/electronics11010037 fatcat:cyc6gtrs6fbeta5tt43qhu3gfi

Learning Hash Codes with Listwise Supervision

Jun Wang, Wei Liu, Andy X. Sun, Yu-Gang Jiang
2013 2013 IEEE International Conference on Computer Vision  
In particular, the ranking information is represented by a set of rank triplets that can be used to assess the quality of ranking.  ...  optimized the search accuracy.  ...  Particularly, for the evaluations by NDCG over the the Hamming ranking in Figures 2(a) , 3(a), 4(a), RSH achieves significant performance gains over all the other methods.  ... 
doi:10.1109/iccv.2013.377 dblp:conf/iccv/WangLSJ13 fatcat:6hrcldgyjrc3vpctdj3eebt6e4

Robust ranking models via risk-sensitive optimization

Lidan Wang, Paul N. Bennett, Kevyn Collins-Thompson
2012 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval - SIGIR '12  
Typically, these techniques increase average effectiveness by devising advanced ranking features and/or by developing sophisticated learning to rank algorithms.  ...  Given that robustness is an important measure that can negatively impact user satisfaction, we present a unified framework for jointly optimizing effectiveness and robustness.  ...  measure M , causing M to be inconsistent.  ... 
doi:10.1145/2348283.2348385 dblp:conf/sigir/WangBC12 fatcat:6sduyl5wyfbxna7byood5whsiq

Plackett-Luce model for learning-to-rank task [article]

Tian Xia, Shaodan Zhai, Shaojun Wang
2019 arXiv   pre-print
However, in real-world applications, state-of-the-art systems are not from list-wise based camp.  ...  This is the first time in the single model level for a list-wise based system to match or overpass state-of-the-art systems in real-world datasets.  ...  [6] introduce it to the learning to rank task by using it to model the probabilistic distribution of a set of documents given a query, where the training is conducted by minimizing the KL distance between  ... 
arXiv:1909.06722v1 fatcat:5rrqzb5vxfbzplvy2jrwgu44wm

Revisiting Performance Measures for Cross-Modal Hashing

Hongya Wang, Shunxin Dai, Ming Du, Bo Xu, Mingyong Li
2022 Proceedings of the 2022 International Conference on Multimedia Retrieval  
In view of this, we propose a new performance measure named Normalized Weighted Discounted Cumulative Gains (NWDCG) by extending Normalized Discounted Cumulative Gains (NDCG) using co-occurrence probability  ...  To verify the effectiveness of NWDCG, we conduct extensive experiments using three popular cross-modal hashing schemes over two publically available datasets.  ...  By analyzing the labels of the dataset, we found that there are different semantic hierarchies among labels, which results in different effects of different labels on the evaluation result.  ... 
doi:10.1145/3512527.3531363 fatcat:tejnwqhdfrdkjoeh55xqso4ae4

FSMRank: Feature Selection Algorithm for Learning to Rank

Han-Jiang Lai, Yan Pan, Yong Tang, Rong Yu
2013 IEEE Transactions on Neural Networks and Learning Systems  
The introduction of feature selection into different learning problems has been proven effective. These facts motivate us to investigate the problem of feature selection for learning to rank.  ...  We propose a joint convex optimization formulation which minimizes ranking errors while simultaneously conducting feature selection.  ...  The first stream optimizes a loss function directly based on IR performance measures [i.e., mean average precision (MAP) or normalized discounted cumulative gain (NDCG)].  ... 
doi:10.1109/tnnls.2013.2247628 pmid:24808475 fatcat:gykgotib3ngerfhaed2twshzd4

Learning to rank from Bayesian decision inference

Jen-Wei Kuo, Pu-Jen Cheng, Hsin-Min Wang
2009 Proceeding of the 18th ACM conference on Information and knowledge management - CIKM '09  
They undoubtedly provide statistically significant improvements over conventional methods; however, from the viewpoint of decision-making, most of them do not minimize the Bayes risk of the IR system.  ...  Recently, several learning to rank methods have been proposed to directly optimize the performance of IR applications in terms of various evaluation measures.  ...  ACKNOWLEDGMENTS This work was supported in part by Taiwan e-Learning and Digital Archives Program (TELDAP) sponsored by the National Science Council of Taiwan under Grant: NSC98-2631-001-013.  ... 
doi:10.1145/1645953.1646058 dblp:conf/cikm/KuoCW09 fatcat:w7ylzdl35ndelld4ftuq4tyqne

Query-level learning to rank using isotonic regression

Zhaohui Zheng, Hongyuan Zha, Gordon Sun
2008 2008 46th Annual Allerton Conference on Communication, Control, and Computing  
We tackle this optimization problem using functional iterative methods where the update at each iteration is computed by solving an isotonic regression problem.  ...  We demonstrate the effectiveness of the proposed method using data sets obtained from a commercial search engine as well as publicly available data.  ...  In the case of multiple levels of judgements, the Normalized Discount Cumulative Gain (NDCG) is used [14] .  ... 
doi:10.1109/allerton.2008.4797684 fatcat:7zrbldomgbexrmhaqvkee7jyzq

Collecting high quality overlapping labels at low cost

Hui Yang, Anton Mityagin, Krysta M. Svore, Sergey Markov
2010 Proceeding of the 33rd international ACM SIGIR conference on Research and development in information retrieval - SIGIR '10  
The proposed scheme collects additional labels only for a subset of training samples, specifically for those that are labeled relevant by a Our experiments show that this labeling schem NDCG of two Web  ...  Our specific focus is performance improvements obtained by using overlapping relevance labels, which collecting multiple human judgments for each training sample.  ...  ACKNOWLEDGEMENT We would like to thank Paul Bennett, Rich Caruana, and Chris Burges for their helpful discussions and feedback, Rangan Majumder for preparing the label sets, and anonymous reviewers for  ... 
doi:10.1145/1835449.1835526 dblp:conf/sigir/YangMSM10 fatcat:snhupbfacffg5caoh5xbal6xgm
« Previous Showing results 1 — 15 out of 299 results