78,861 Hits in 6.8 sec

Evaluation Measures for Relevance and Credibility in Ranked Lists [article]

Christina Lioma and Jakob Grue Simonsen and Birger Larsen
2017 arXiv   pre-print
Motivated by the above, we present two novel types of evaluation measures that are designed to measure the effectiveness of both relevance and credibility in ranked lists of retrieval results.  ...  Whereas technology is in place for filtering information according to relevance and/or credibility, no single measure currently exists for evaluating the accuracy or precision (and more generally effectiveness  ...  Empirical Evaluation The goal is to determine how good our measures are at evaluating both relevance and credibility in ranked lists.  ... 
arXiv:1708.07157v1 fatcat:la33o5i2onalrggedjzhwcllp4

Finding Credible Information Sources in Social Networks Based on Content and Social Structure

Kevin R. Canini, Bongwon Suh, Peter L. Pirolli
2011 2011 IEEE Third Int'l Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third Int'l Conference on Social Computing  
A task of primary importance for social network users is to decide whose updates to subscribe to in order to maximize the relevance, credibility, and quality of the information received.  ...  To address this problem, we conducted an experiment designed to measure the extent to which different factors in online social networks affect both explicit and implicit judgments of credibility.  ...  By combining a basic text search with a social ranking technique and topic modeling analysis, the algorithm generates a ranked list of relevant, trusted, and credible Twitter users for any given topic.  ... 
doi:10.1109/passat/socialcom.2011.91 dblp:conf/socialcom/CaniniSP11 fatcat:4rxwj7wzjreafa53bjsxht6h5y

UWaterlooMDS at the TREC 2019 Decision Track

Mustafa Abualsaud, Fuat C. Beylunioglu, Mark D. Smucker, P. Robert Duimering
2019 Text Retrieval Conference  
In this report, we discuss the experiments we conducted for the TREC 2019 Decision Track. This year, our goal was to investigate the effect of document credibility on the quality of automatic runs.  ...  To address credibility, we combined scores from a spam classifier and a credibility classifier trained to detect non-trustworthy websites.  ...  ACKNOWLEDGMENTS This work was supported in part by the Natural Sciences and Engineering Research Council of Canada (RGPIN-2014-03642), and in part by the University of Waterloo.  ... 
dblp:conf/trec/AbualsaudBSD19 fatcat:h5y5ffiwjbcq3hnisqig6dm2ua

Imcube @ MediaEval 2015 Retrieving Diverse Social Images Task: Multimodal Filtering and Re-ranking

Sebastian Schmiedeke, Pascal Kelm, Lutz Goldmann
2015 MediaEval Benchmarking Initiative for Multimedia Evaluation  
Therefore, we propose a multi-modal approach for filtering and re-ranking in order to improve the relevancy and diversity of the returned list of ranked images.  ...  This task addresses the problem of result diversification in the context of social photo retrieval where the results of a query should contain relevant but diverse items.  ...  way that the most relevant and diverse images are returned in a ranked list of up to 50 images.  ... 
dblp:conf/mediaeval/SchmiedekeKG15 fatcat:dvw54qu5ffgqpmbfijdkmhgsmq

Recod @ MediaEval 2015: Diverse Social Images Retrieval

Rodrigo Tripodi Calumby, Iago B. A. do C. Araujo, Vinícius Pereira Santana, Javier A. V. Muñoz, Otávio Augusto Bizetto Penatti, Lin Tzy Li, Jurandy Almeida, Giovani Chiachia, Marcos André Gonçalves, Ricardo da Silva Torres
2015 MediaEval Benchmarking Initiative for Multimedia Evaluation  
We proposed a multimodal approach and exploited image metadata and user credibility information.  ...  Our proposal is based on irrelevant image filtering, reranking, rank aggregation, and diversity promotion.  ...  Reranking and Aggregation For improving the original list ranking, we explored visual, textual, credibility, and geographic ranking.  ... 
dblp:conf/mediaeval/CalumbyASMPLACG15 fatcat:hkx46us3gbhyvmdocrlhcggqmy

Learning to rank for Consumer Health Search

Hua Yang, Xiaoming Liu, Binbin Zheng, Guan Yang
2021 Conference and Labs of the Evaluation Forum  
This paper presents the work of the Zhongyuan University of Technology participating in Subtask 1. It explores the use of learning to rank techniques in consumer health search.  ...  The best four models are used to re-rank the documents and four runs are submitted to the subtask.  ...  Evaluation Measures The task takes into account 3 dimensions in the relevance evaluation: topical relevance, understandability, and credibility.  ... 
dblp:conf/clef/YangLZY21 fatcat:27em2qobxnetdlywwj3vgpwlb4

Searching for Authoritative Documents in Knowledge-Base Communities

G. Alan Wang, Jian Jiao, Weiguo Fan
2009 International Conference on Information Systems  
In this paper we study the problem of finding authoritative documents for user queries within a knowledge-based community.  ...  We design two ranking strategies that blend these two sources of evidence with the content-based relevance judgment.  ...  The 11 VB questions listed in Table 3 were used as our search queries when evaluating our proposed search methods.  ... 
dblp:conf/icis/WangJF09 fatcat:rf5oqv5dura5ppn7xykoa5nlbe

CFM@MediaEval 2017 Retrieving Diverse Social Images Task via Re-ranking and Hierarchical Clustering

Liang Peng, Yi Bin, Xiyao Fu, Jie Zhou, Yang Yang, Heng Tao Shen
2017 MediaEval Benchmarking Initiative for Multimedia Evaluation  
Our approach achieves a good tradeoff between relevance and diversity and a result in F1@20 of 0.6533 for the employed test data.  ...  This paper presents an approach based on re-ranking and hierarchical clustering (HC) for MediaEval 2017 Retrieving Diverse Social Images Task.  ...  Clustering After re-ranking, we obtain a more relevant rank list of the initial N images for each topic (N = 300 in the task).  ... 
dblp:conf/mediaeval/PengBFZYS17 fatcat:rjxhotpo6vhqblqknh6uzxul7y

#CycloneGaja-Rank based Credibility Analysis System in social media during the crisis

P. Suthanthira Devi, S. Karthika
2019 Procedia Computer Science  
Twitter plays an instrumental role in public and political conversations related to the particular crisis events.  ...  This research work aims to rank the credibility of the tweets associated with the Natural Disasters and identifies the tweets that spread fake information.  ...  It measures logically down weights the gain received from the Twitter data and examine the long ranked result list by adjusting the discounting value.  ... 
doi:10.1016/j.procs.2020.01.064 fatcat:c4vf52omlfgwpjkx36wlwsrokm

Enhancing credibility judgment of web search results

Yusuke Yamamoto, Katsumi Tanaka
2011 Proceedings of the 2011 annual conference on Human factors in computing systems - CHI '11  
In this paper, we propose a system for helping users to judge the credibility of Web search results and to search for credible Web pages.  ...  Moreover, ranking algorithms of the conventional Web search engines are often based on relevance and popularity of Web pages.  ...  ACKNOWLEDGEMENTS This work was supported in part by the following projects and institutions: Grants-in-Aid for Scientific Research (No. 18049041) from MEXT of Japan, a Kyoto University GCOE Program entitled  ... 
doi:10.1145/1978942.1979126 dblp:conf/chi/YamamotoT11 fatcat:ard4veaqbrcw7ia4at6cgxjvr4

Rank Fusion and Multimodal Per-topic Adaptiveness for Diverse Image Retrieval

Rodrigo Tripodi Calumby, Iago Breno Alves do Carmo Araujo, Felipe Souza Cordeiro, Fabiana Bertoni, Sérgio D. Canuto, Fabiano Belém, Marcos André Gonçalves, Ícaro C. Dourado, Javier Munoz, Lin Li, Ricardo da Silva Torres
2017 MediaEval Benchmarking Initiative for Multimedia Evaluation  
Our proposal for relevance improvement relies on text and credibility-based reranking and rank aggregation.  ...  We applied Genetic Programming and Genetic Algorithm-based approaches for combining textual, visual, and user credibility information.  ...  ACKNOWLEDGMENTS The authors thank FAPESP, FAPEMIG, CAPES, CNPq, and FAPESB (grant #4098/2016) for partially funding this work.  ... 
dblp:conf/mediaeval/CalumbyACBCBGDM17 fatcat:gdhyped7zbe4hdpit6ilacnl54

An Overview on Evaluation Labs and Open Issues in Health-related Credible Information Retrieval

Rishabh Upadhyay, Gabriella Pasi, Marco Viviani
2021 Italian Information Retrieval Workshop  
Faced with the problem of widespread health misinformation, in recent years credibility is being considered as one of the important dimensions for health-related information access and retrieval.  ...  To encourage research in this field, a couple of evaluation labs have been recently set up to provide large test collections, baselines, and evaluation metrics to interested researchers.  ...  Acknowledgments This work is supported by the EU Horizon 2020 Research and Innovation Programme under the Marie Skłodowska-Curie Grant Agreement No 860721 -DoSSIER: "Domain Specific Systems for Information  ... 
dblp:conf/iir/UpadhyayPV21 fatcat:f7kqbdxe4zcwpjitjs55gcgpzy

Credible Mechanism for More Reliable Search Engine Results

Mohammed Abdel Razek
2015 International Journal of Information Technology and Computer Science  
It presents a design and development of a credible mechanism that searches Web search engine and then ranks sites according to its reliability.  ...  This paper investigates aspects affecting on the Websites credibility and then uses them along with dominant meaning of the query for improving information retrieval capabilities and to effectively manage  ...  In Fig.6 , we merge recall and precision into a single overall measure to clarify the impact of using credible value for ranking Google results vs. Google ranking.  ... 
doi:10.5815/ijitcs.2015.03.02 fatcat:boxstoth25h6hkcggtskikcj2u

UPV at TREC Health Misinformation Track 2021 Ranking with SBERT and Quality Estimators [article]

Ipek Baris Schlicht and Angel Felipe Magnossão de Paula and Paolo Rosso
2021 arXiv   pre-print
We use a BM25 and a domain-specific semantic search engine for retrieving initial documents. Later, we examine a health news schema for quality assessment and apply it to re-rank documents.  ...  We merge the scores from the different components by using reciprocal rank fusion. Finally, we discuss the results and conclude with future works.  ...  Also, we would like to thank Ronak Pradeep for his help in providing the subset of the index corpus to us.  ... 
arXiv:2112.06080v1 fatcat:l3ymmqe37vefnnpm43mgppyygq

Fight Against COVID-19 Misinformation via Clustering-Based Subset Selection Fusion Methods

Yidong Huang, Qiuyu Xu, Shengli Wu, Christopher Nugent, Adrian Moore
2022 European Conference on Information Retrieval  
Especially, a clustering-based approach is proposed for selecting a subset of systems from all available ones for finding relevant, credible, and correct documents.  ...  In such a situation, how to provide useful and correct information to the public is not straightforward.  ...  respectively any valid relevance, correctness, and credibility evaluation measures.  ... 
dblp:conf/ecir/HuangXWN022 fatcat:lg5bxss2grdslgwqhgo5u6fzuq
« Previous Showing results 1 — 15 out of 78,861 results