Filters








1,982 Hits in 3.8 sec

The TREC-like evaluation of music IR systems

J. Stephen Downie
2003 Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval - SIGIR '03  
The proposed research tasks are based upon expert opinion garnered from members of the Information Retrieval (IR), MDL and MIR communities with regard to the construction and implementation of scientifically  ...  This poster reports upon the ongoing efforts being made to establish TREC-like and other comprehensive evaluation paradigms within the Music IR (MIR) and Music Digital Library (MDL) research communities  ...  TREC-like is used deliberately as music retrieval presents some important differences to text retrieval and our evaluation tests must take these into account (see Major Research Questions below).  ... 
doi:10.1145/860435.860547 dblp:conf/sigir/Downie03 fatcat:rkodsdbojjfs3lqedm72ws52ca

The TREC-like evaluation of music IR systems

J. Stephen Downie
2003 Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval - SIGIR '03  
The proposed research tasks are based upon expert opinion garnered from members of the Information Retrieval (IR), MDL and MIR communities with regard to the construction and implementation of scientifically  ...  This poster reports upon the ongoing efforts being made to establish TREC-like and other comprehensive evaluation paradigms within the Music IR (MIR) and Music Digital Library (MDL) research communities  ...  TREC-like is used deliberately as music retrieval presents some important differences to text retrieval and our evaluation tests must take these into account (see Major Research Questions below).  ... 
doi:10.1145/860500.860547 fatcat:gszllm57t5ckvj2emvrzdczevq

An advanced search engine for patent analytics in medicinal chemistry

Emilie Pasche, Julien Gobeill, Douglas Teodoro, Arnaud Gaudinat, Dina Vishnykova, Christian Lovis, Patrick Ruch
2012 Studies in Health Technology and Informatics  
Patent collections contain an important amount of medical-related knowledge, but existing tools were reported to lack of useful functionalities.  ...  Nonetheless, TWINC performed well during the Chemathlon task of the PatOlympics competition and experts appreciated its usability.  ...  Ad hoc search is evaluated using a set of six topics ( Figure 1 ) defined by means of natural language by TREC evaluators for the TS task of TREC-Chem 2011.  ... 
pmid:22874181 fatcat:bkeo5unbjbejdmn4celyie43iy

A user-friendly tool for medical-related patent retrieval

Emilie Pasche, Julien Gobeill, Douglas Teodoro, Arnaud Gaudinat, Dina Vishnyakova, Christian Lovis, Patrick Ruch
2012 Studies in Health Technology and Informatics  
Health-related information retrieval is complicated by the variety of nomenclatures available to name entities, since different communities of users will use different ways to name a same entity.  ...  Nonetheless, TWINC performed well during the PatOlympics competition and was appreciated by intellectual property experts. This result should be balanced by the limited evaluation sample.  ...  The European Commission, Directorate General Information Society and Media, Brussels, is not liable for any use that may be made of the information contained therein.  ... 
pmid:22491124 fatcat:2ihzgg4s2jd2nmlqebowh2tvba

The SIGIR 2008 workshop on future challenges in expertise retrieval (fCHER)

Krisztian Balog
2008 SIGIR Forum  
[8] , entitled Evaluating Relation Retrieval for Entities and Experts, explores entity retrieval, a research area that is closely related to expert finding.  ...  And what sets experts and expertise apart? The TREC Enterprise Search track will be put on hold in 2009 due to the evaluation challenges discussed before.  ... 
doi:10.1145/1480506.1480515 fatcat:7z2uhqyipraojnnylwh7untupm

Entity information management in complex networks

Yi Fang
2010 Proceeding of the 33rd international ACM SIGIR conference on Research and development in information retrieval - SIGIR '10  
In the recent years, entity retrieval especially expert search has attracted much attention in the IR community while many other EIM problems have been rarely investigated.  ...  The research is an intersection of content based information retrieval and complex network analysis, which deals with both unstructured text data and structured networks.  ...  The entity workshops in TREC 2008 and TREC 2009 briefly discussed these two tasks 3 , but they have not been included in the TREC Entity track yet.  ... 
doi:10.1145/1835449.1835690 dblp:conf/sigir/Fang10 fatcat:v4zgq7fofjbk7jdq2y6j42xpse

Searching for Expertise: Experiments with the Voting Model

C. Macdonald, I. Ounis
2008 Computer journal  
The evaluation results show that the voting paradigm is very effective, without using any collection-specific heuristics.  ...  We see the problem of ranking experts as a voting problem, which we model using adaptations of data fusion techniques.  ...  Indeed, the median MAP THE VOTING MODEL FOR EXPERT SEARCH Page 7 of 20 for the TREC 2005 task was 0.1402 [3], while for TREC 2006 the median MAP was 0.3412 TABLE 2 . 2 Evaluation using the expert  ... 
doi:10.1093/comjnl/bxm112 fatcat:rfujhcaiovdfrjnhavinikpvne

Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR'17)

Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
2018 SIGIR Forum  
Organizers of four of the TREC 2017 tracks were invited to present at the workshop on how these IR tasks may be suitable for evaluating recent data-hungry neural approaches.  ...  The second SIGIR workshop on neural information retrieval (Neu-IR'17) took place on August 11, 2017, in Tokyo, Japan.  ...  TREC track Speaker Website Complex Answers Retrieval Laura Dietz http://trec-car.cs.unh.edu/ Real-Time Summarization Jimmy Lin http://trecrts.github.io/ Dynamic Domain Grace Hui Yang http://trec-dd.org  ... 
doi:10.1145/3190580.3190603 fatcat:23pwl6jtnbdh3ifxexlwqozdhu

The Scientific Evaluation of Music Information Retrieval Systems: Foundations and Future

J. Stephen Downie
2004 Computer Music Journal  
This paper outlines the findings-to-date of a project to assist in the efforts being made to establish a TREC-like evaluation paradigm within the Music Information Retrieval (MIR) research community.  ...  The findings and recommendations are based upon expert opinion garnered from members of the Information Retrieval (IR), Music Digital Library (MDL) and MIR communities with regard to the construction and  ...  The idea that the TREC-like evaluation scenario not be the sole evaluation approach used was iterated in Reiss and Sandler (2002b) .  ... 
doi:10.1162/014892604323112211 fatcat:whd4eo2xb5a6damjrr6oskdv2y

A User-Oriented Model for Expert Finding [chapter]

Elena Smirnova, Krisztian Balog
2011 Lecture Notes in Computer Science  
We use the distance between the user and an expert in a social network to estimate contact time, and consider various social graphs, based on organizational hierarchy, geographical location, and collaboration  ...  Using a realistic test set, created from interactions of employees with a university-wide expert search engine, we demonstrate substantial improvements over a state-of-the-art baseline on all retrieval  ...  Baseline Model Probabilistic approaches to expertise retrieval have been both popular and successful at the TREC Enterprise track [3, 8, 11, 28] .  ... 
doi:10.1007/978-3-642-20161-5_58 fatcat:xjkwzrs37fdcvhmcwht3i345cu

CITREC: An Evaluation Framework for Citation-Based Similarity Measures based on TREC Genomics and PubMed Central

Bela Gipp, Norman Meuschke, Mario Lipinski
2015 Zenodo  
The first gold standard uses the Medical Subject Headings (MeSH) thesaurus and the second uses the expert relevance feedback that is part of the TREC Genomics collection to gauge similarity.  ...  CITREC prepares the data from the PubMed Central Open Access Subset and the TREC Genomics collection for a citation-based analysis and provides tools necessary for performing evaluations of similarity  ...  Since a document can contain more than one relevant passage, the experts judged passages from 11,638 documents. 𝑝(𝑜 3 ) = 𝑝(𝑜 4𝑎 ) = 𝑝(𝑜 4𝑏 ) = 𝑝(𝑜 8 ) = 𝑠𝑖𝑚(𝑜 The TREC Genomics track evaluated  ... 
doi:10.5281/zenodo.3547372 fatcat:3izqxgttdncqdhige5oclldqcm

Searching distributed collections with inference networks

James P. Callan, Zhihong Lu, W. Bruce Croft
1995 Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '95  
The use of information retrieval systems in networked environments raises a new set of issues that have received little attention.  ...  . 1 Personal communication from a commercial retrieval service.  ...  Acknowledgements This research was partially supported by the NSF Center for Intelligent Information Retrieval at the University of Massachusetts, Amherst.  ... 
doi:10.1145/215206.215328 dblp:conf/sigir/CallanLC95 fatcat:nkowgsvvvzelhmjsdy4w3ieeim

Searching Distributed Collections With Inference Networks

James P. Callan, Zhihong Lu, W. Bruce Croft
2017 SIGIR Forum  
The use of information retrieval systems in networked environments raises a new set of issues that have received little attention.  ...  . 1 Personal communication from a commercial retrieval service.  ...  Acknowledgements This research was partially supported by the NSF Center for Intelligent Information Retrieval at the University of Massachusetts, Amherst.  ... 
doi:10.1145/3130348.3130363 fatcat:cezy4t4jonhgbld25jput4qliu

SLEDGE: A Simple Yet Effective Baseline for COVID-19 Scientific Knowledge Search [article]

Sean MacAvaney, Arman Cohan, Nazli Goharian
2020 arXiv   pre-print
We train the model on a general-domain answer ranking dataset, and transfer the relevance signals to SARS-CoV-2 for evaluation.  ...  We observe SLEDGE's effectiveness as a strong baseline on the TREC-COVID challenge (topping the learderboard with an nDCG@10 of 0.6844).  ...  We evaluate the performance of SLEDGE using the TREC-COVID Information Retrieval Challenge dataset (round 1) (Roberts et al., 2020) .  ... 
arXiv:2005.02365v3 fatcat:i5lmt5jvzbg7dojiaf2zjhg2t4

Generative modeling of persons and documents for expert search

Pavel Serdyukov, Djoerd Hiemstra, Maarten Fokkinga, Peter M. G. Apers
2007 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '07  
We present the theoretically-based probabilistic algorithm which models retrieved documents as mixtures of expert candidate language models.  ...  In this paper we address the task of automatically finding an expert within the organization, known as the expert search problem.  ...  CONCLUSIONS We presented a new generative model-based method for expert finding and evaluated it using the TREC Enterprise collection.  ... 
doi:10.1145/1277741.1277929 dblp:conf/sigir/SerdyukovHFA07 fatcat:or3m6odxifhztk2ta3yqks3jwy
« Previous Showing results 1 — 15 out of 1,982 results