96 Hits in 5.7 sec

Users and Assessors in the Context of INEX: Are Relevance Dimensions Relevant? [article]

Jovan Pehcevski, Anne-Marie Vercoustre
2005 arXiv   pre-print
are perceived in the same way by both the assessor and the users.  ...  We argue that the two INEX relevance dimensions, Exhaustivity and Specificity, are not orthogonal dimensions; indeed, an empirical analysis of each dimension reveals that the grades of the two dimensions  ...  Acknowledgements We thank Saied Tahaghoghi and the anonymous reviewers for providing useful comments on earlier drafts of this paper.  ... 
arXiv:cs/0507069v1 fatcat:ur3vkvfekvfbvboajed6kne5re

A Study of the Assessment of Relevance for the INEX'02 Test Collection [chapter]

Gabriella Kazai, Sherezad Masood, Mounia Lalmas
2004 Lecture Notes in Computer Science  
We investigate possible assessment trends and inconsistencies within the collected relevance assessments of the INEX'02 test collection in order to provide a critical analysis of the employed relevance  ...  criterion and assessment procedure for the evaluation of content-oriented XML retrieval approaches.  ...  This is addressed in INEX'03 by means of rigorous checks being implemented in the assessment tool, where assessors are required to assess all relevant ascendant and descendant components.  ... 
doi:10.1007/978-3-540-24752-4_22 fatcat:t3wahcljgvf6rilqzflqnfa2p4

Report on INEX 2008

Gianluca Demartin, Gabriella Kazai, Marijn Koolen, Monica Landoni, Ragnar Nordlie, Nils Pharo, Ralf Schenkel, Martin Theobald, Andrew Trotman, Arjen P. de Vries, Alan Woodley, Ludovic Denoye (+8 others)
2009 SIGIR Forum  
This paper reports on the INEX 2008 evaluation campaign, which consisted of a wide range of tracks: Ad hoc, Book, Efficiency, Entity Ranking, Interactive, QA, Link the Wiki, and XML Mining. • Link-the-Wiki  ...  and clustering of semi-structured documents.  ...  in Context, Best in Context). • Book Track Investigating techniques to support users in reading, searching, and navigating full texts of digitized books. • Efficiency Track Investigating the trade-off  ... 
doi:10.1145/1670598.1670603 fatcat:zoxbecrybrf63fg54g4kt7w7na

Overview of INEX 2005 [chapter]

Saadia Malik, Gabriella Kazai, Mounia Lalmas, Norbert Fuhr
2006 Lecture Notes in Computer Science  
Since 2002, INEX has been working towards the goal of establishing an infrastructure, in the form of a large XML test collection and appropriate scoring methods, for the evaluation of content-oriented  ...  This paper provides an overview of the work carried out as part of INEX 2005.  ...  2006 will have two new tracks: User case studies and XML entity ranking tracks. In addition, a new test collection will be used, based on the Wikipedia project.  ... 
doi:10.1007/978-3-540-34963-1_1 fatcat:dh4n6k3tuvdgnm4hebiebtvisa

The use case track at INEX 2006

Nils Pharo, Andrew Trotman
2007 SIGIR Forum  
In the present article we present a summary of the discussions and findings of the track.  ...  The use case track is a new track at INEX and represents an attempt at identifying examples of how XML IR systems can be exploited by end-users for various purposes.  ...  An example of a relevant-in-context use case is: The Relevant in Context (and Fetch & Browse) Use Case • The user needs factual information to solve a dispute.  ... 
doi:10.1145/1273221.1273227 fatcat:uf2qkrqftvazphcs3ydi2ktbce

Evaluating relevant in context

Jaap Kamps, Mounia Lalmas, Jovan Pehcevski
2007 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '07  
Our main research question is: how to evaluate the Relevant in Context task?  ...  The Relevant in Context retrieval task is document or article retrieval with a twist, where not only the relevant articles should be retrieved but also the relevant information within each article (captured  ...  This assumes that users consider the article as the most natural unit of retrieval, and prefer an overview of relevance in their context.  ... 
doi:10.1145/1277741.1277890 dblp:conf/sigir/KampsLP07 fatcat:wfsfda3vxremhkzhrhonm4tzke

INEX Tweet Contextualization task: Evaluation, results and lesson learned

Patrice Bellot, Véronique Moriceau, Josiane Mothe, Eric SanJuan, Xavier Tannier
2016 Information Processing & Management  
Microblogging platforms such as Twitter are increasingly used for on-line client and market analysis. This motivated the proposal of a new track at CLEF INEX lab of Tweet Contextualization.  ...  informativeness of produced contexts or summaries.  ...  We would like to acknowledge the French ANR agency for their support through the CAAS-Contextual Analysis and Adaptive Search project (ANR-10-CORD-001 01).  ... 
doi:10.1016/j.ipm.2016.03.002 fatcat:7h64buwylvhefbcarhxeusjd5q

Overview of the INEX 2011 Books and Social Search Track [chapter]

Marijn Koolen, Gabriella Kazai, Jaap Kamps, Antoine Doucet, Monica Landoni
2012 Lecture Notes in Computer Science  
The goal of the INEX 2011 Books and Social Search Track is to evaluate approaches for supporting users in reading, searching, and navigating book metadata and full texts of digitized books.  ...  We report on the setup and the results of the track. 5  ...  A crib sheet is provided to participants as a tool to define the user tasks to evaluate, providing a narrative describing the scenario(s) of use for the books in context, including factors affecting user  ... 
doi:10.1007/978-3-642-35734-3_1 fatcat:c4bz33oixzdrvlvlfbrhkgplvu

Sound and complete relevance assessment for XML retrieval

Benjamin Piwowarski, Andrew Trotman, Mounia Lalmas
2008 ACM Transactions on Information Systems  
The different methods of gathering assessments at INEX are discussed and contrasted. The highlighting method is shown to be the most reliable of the methods.  ...  In information retrieval research, comparing retrieval approaches requires test collections consisting of documents, user requests and relevance assessments.  ...  Acknowledgements The INEX initiative is funded by DELOS, a network of excellence for digital libraries.  ... 
doi:10.1145/1416950.1416951 fatcat:yhpga73j6famxfmuyp6gviw3ma

Providing consistent and exhaustive relevance assessments for XML retrieval evaluation

Benjamin Piwowarski, Mounia Lalmas
2004 Proceedings of the Thirteenth ACM conference on Information and knowledge management - CIKM '04  
This paper describes the methodology adopted in INEX (the INitiative for the Evaluation of XML Retrieval) to ensure consistent and exhaustive relevance assessments.  ...  This is because XML documents are composed of nested components that cannot be considered independent in terms of relevance.  ...  Acknowledgment We would like to thank the reviewers for their excellent and helpful comments, and also G. Kazai  ... 
doi:10.1145/1031171.1031246 dblp:conf/cikm/PiwowarskiL04 fatcat:e3rw56o5kfcj7gwqwntmxffb6m

A Bayesian Framework for XML Information Retrieval: Searching and Learning with the INEX Collection

Benjamin Piwowarski, Patrick Gallinari
2005 Information retrieval (Boston)  
This is the focus of the paper: we propose here to train the structured Bayesian Network model using a cross-entropy training criterion. Results are presented on the INEX corpus of XML documents.  ...  This model is based on Bayesian networks whose conditional probabilities are learnt from a labelled collection of structured documents-which is composed of documents, queries and their associated assessments  ...  The conceptualisation of SIR is clearly not the same for the two assessors. Note that instructions for INEX are in accordance Figure 9 .  ... 
doi:10.1007/s10791-005-0751-6 fatcat:j3h342fpz5gqpfd4tiprqmacu4

Worker types and personality traits in crowdsourcing relevance labels

Gabriella Kazai, Jaap Kamps, Natasa Milic-Frayling
2011 Proceedings of the 20th ACM international conference on Information and knowledge management - CIKM '11  
Using data collected from workers engaged in the crowdsourced evaluation of the INEX 2010 Book Track Prove It task, we relate the worker types to label accuracy and personality trait information along  ...  the 'Big Five' personality dimensions.  ...  Personality traits have been shown to relate to user behavior in the context of e-commerce, e.g., [8] , social media, e.g., [5] and web browsing and searching behavior, e.g., [15] .  ... 
doi:10.1145/2063576.2063860 dblp:conf/cikm/KazaiKM11 fatcat:q444dxg6frhadpysk4t2ettrqe

Report on the SIGIR 2006 workshop on XML element retrieval methodology

Andrew Trotman, Shlomo Geva
2006 SIGIR Forum  
This report outlines the events of the workshop and summarized the major outcomes.  ...  On the 10 th of August 2006 the SIGIR 2006 Workshop on XML Element Retrieval Methodology was held as part of SIGIR in Seattle, Washington, USA. Six papers were presented in four sessions.  ...  The University of Otago is hosting the workshop proceedings which are online (  ... 
doi:10.1145/1189702.1189709 fatcat:pquhmrs4xjbupdgxgepbiwpj2i

Advances in XML retrieval

Norbert Fuhr, Mounia Lalmas
2007 Proceedings of the 2006 international workshop on Research issues in digital libraries - IWRIDL '06  
Then we present a new framework for the different views on XML retrieval, where we distinguish between the structural and the content dimension; in this space, current activities are located as well as  ...  We given a survey over the INEX initiative, which focuses on the evaluation of content based access to XML documents. First, we describe the test setting and the various tracks of INEX.  ...  We briefly described the test setting and the various tracks of INEX.  ... 
doi:10.1145/1364742.1364763 dblp:conf/iwridl/FuhrL06 fatcat:r75prvrnrzbpbisc5sdgqjblcu

Investigating the exhaustivity dimension in content-oriented XML element retrieval evaluation

Paul Ogilvie, Mounia Lalmas
2006 Proceedings of the 15th ACM international conference on Information and knowledge management - CIKM '06  
A continuous discussion in INEX has been whether such a sophisticated definition of relevance, and in particular the exhaustivity dimension, was needed.  ...  INEX, the evaluation initiative for content-oriented XML retrieval, has since its establishment defined the relevance of an element according to two graded dimensions, exhaustivity and specificity.  ...  Any opinions, findings, conclusions or recommendations expressed in this paper are the authors', and do not necessarily reflect those of the sponsor.  ... 
doi:10.1145/1183614.1183631 dblp:conf/cikm/OgilvieL06 fatcat:fj2ik7me3rdzpm74nzycrdwsha
« Previous Showing results 1 — 15 out of 96 results