77,527 Hits in 6.1 sec

ELDAMeth: An agent-oriented methodology for simulation-based prototyping of distributed agent systems

Giancarlo Fortino, Wilma Russo
2012 Information and Software Technology  
A simple yet effective case study in the distributed information retrieval domain is used to illustrate the proposed methodology.  ...  ELDAMeth can be used both standalone for the modeling and evaluation of DAS and coupled with other agent-oriented methodologies for enhancing them with simulation-based validation.  ...  ACKNOWLEDGMENT Authors wish to thank A. Garro, S. Mascillaro, G. Mazzitelli, and F. Rango for useful ideas, discussions and implementation efforts supporting the ELDAMeth project.  ... 
doi:10.1016/j.infsof.2011.08.006 fatcat:5z2jsnaamfhsjeyeqnprbghvqm

An analysis of two approaches in information retrieval: From frameworks to study designs

Kalervo Järvelin
2007 Journal of the American Society for Information Science and Technology  
There is a well-known gap between Systems-oriented IR and User-oriented IR, which Cognitive IR seeks to bridge.  ...  Thus each approach is capable of contributing very differently to understanding and developing information access. The article also discusses integrating the approaches at the study design level.  ...  The latter proposes work task descriptions and an evaluation package for Interactive Information Retrieval (IIR) including novel effectiveness measures and guidelines for setting up experiments.  ... 
doi:10.1002/asi.20589 fatcat:g2cbl5um6fhxfaqnynecn6rxbm

Conversational AI from an Information Retrieval Perspective: Remaining Challenges and a Case for User Simulation

Krisztian Balog
2021 Biennial Conference on Design of Experimental Search & Information Retrieval Systems  
This paper highlights challenges that render both offline and online evaluation methodologies unsuitable for this problem, and discusses the use of user simulation as a viable solution.  ...  There remains a disconnect between current efforts and truly conversational information access systems.  ...  We, however, avoid using the term chatbot, which has a different meaning in industrial and academic contexts; in the former case it refers to a task-oriented system, while in the latter it means a non-task-oriented  ... 
dblp:conf/desires/Balog21 fatcat:5iwrv7mgt5asjeo2i4rsockg6u

Physicists' information tasks

Marianne Lykke, Peter Ingwersen, Toine Bogers, Haakon Lund, Birger Larsen
2010 Proceeding of the third symposium on Information interaction in context - IIiX '10  
We analyze 1) the main purpose of the information task, 2) which and how many search facets were used to describe the tasks, 3) what semantic categories were used to express the search facets, and 4) retrieval  ...  The results indicate effect of length and, in particular, of task purpose on retrieval performance of different document description levels that should be examined further.  ...  We have developed a test collection that supports system-driven as well as user-oriented evaluation, based on genuine work task situations, real information tasks, and non-binary relevance judgements  ... 
doi:10.1145/1840784.1840837 dblp:conf/iiix/LykkeIBLL10 fatcat:hsgltjpfg5drzarj5xurjraefu

Evaluation in Contextual Information Retrieval

Lynda Tamine, Mariam Daoud
2018 ACM Computing Surveys  
While evaluation is central and has a long history in information retrieval, it faces the big challenge of designing an appropriate methodology that embeds the context into evaluation settings.  ...  More specifically, t his s urvey a rticle a ims t o fill tw o ma in ga ps in th e li terature: Fi rst, it provides a critical summary and comparison of existing contextual information retrieval evaluation  ...  A system is evaluated with respect to its ability to searching among secrets, in other words, retrieving both relevant and non-sensitive information.  ... 
doi:10.1145/3204940 fatcat:qzvhpa6rlvhh5hrsbblhiulwvy

The development of a method for the evaluation of interactive information retrieval systems

Pia Borlund, Peter Ingwersen
1997 Journal of Documentation  
The paper describes the ideas and assumptions underlying the development of a new method for the evaluation and testing of interactive information retrieval (IR) systems, and reports on the initial tests  ...  The method takes into account the dynamic nature of information needs which are assumed to develop over time for the same user, a variability which is presumed to be strongly connected to the processes  ...  ACKNOWLEDGEMENTS The authors wish to express their gratitude to Information Officer Hanne Balch Petersen and Professor Micheline Beaulieu for their conceptual and methodological support during the project  ... 
doi:10.1108/eum0000000007198 fatcat:sabk5fnc3jca7i6a57ypnte4ry

Interactivity at the Text Retrieval Conference (TREC)

William Hersh, Paul Over
2001 Information Processing & Management  
The papers re¯ect an interest in the process of interactive searching as well as results in the observation, measurement, and evaluation of a human searcher interacting with a search system and data ±  ...  Most of the research carried out within TREC has involved testing information retrieval (IR) systems in a fully automatic setting.  ...  These studies give us a glimpse into the complexity of interactive retrieval evaluation.  ... 
doi:10.1016/s0306-4573(00)00052-2 fatcat:kc2huxdr4jfenlnrynginfzaaq

User-oriented evaluation of a medical image retrieval system for radiologists

Dimitrios Markonis, Markus Holzer, Frederic Baroz, Rafael Luis Ruiz De Castaneda, Célia Boyer, Georg Langs, Henning Müller
2015 International Journal of Medical Informatics  
Purpose: This article reports the user-oriented evaluation of a text-and content-based medical image retrieval system.  ...  Moreover, the methodology presented as well as the discussion on the limitations and challenges of such studies can be useful for useroriented medical image retrieval evaluation, as user-oriented evaluation  ...  Conclusion User-oriented design and evaluation is necessary for developing applications that correspond to realistic information needs and can have an impact on medical practice.  ... 
doi:10.1016/j.ijmedinf.2015.04.003 pmid:26163487 fatcat:ydfkgf32prb43icdmeulw2vvpu

Overview of INEX 2004 [chapter]

Saadia Malik, Mounia Lalmas, Norbert Fuhr
2005 Lecture Notes in Computer Science  
Evaluating the effectiveness of XML retrieval systems requires a test collection (XML documents, tasks/topics, and relevance judgements) where the relevance assessments are provided according to a relevance  ...  The aim of this initiative is to provide means, in the form of large testbeds and appropriate scoring methods, for the evaluation of content-oriented retrieval of XML documents.  ...  Within the ad hoc retrieval task, INEX 2004 defined the following two sub-tasks: CO and VCAS. The CO task stands for content-oriented XML retrieval using CO queries.  ... 
doi:10.1007/11424550_1 fatcat:mg7biirdufbj3chkoy6cougkgq

A task completion framework to support single-interaction IR research

Anton van der Vegt, Guido Zuccon, Bevan Koopman, Peter Bruza
2018 Journal of Documentation  
Approach: A first principles approach is employed to develop a task-oriented conceptual model, called Bridging Information Retrieval (BIR).  ...  Originality: This work contributes a novel task-oriented IR conceptual model and evaluation framework, both centred around the concept of BI for siIR.  ...  This scenario involves multiple user interactions with the IR system, i.e., After the user commences the search with a query (interaction 1), the system retrieves a list of information objects which the  ... 
doi:10.1108/jd-09-2017-0128 fatcat:fjv63pot35efrpyga7eyvrghiq

Assessing Language Learners' Free Productive Vocabulary with Hidden-task-oriented Dialogue Systems

Dolça Tellols, Takenobu Tokunaga, Hilofumi Yamamoto
2020 International Conference on Intelligent User Interfaces  
Because these systems do not fall into the existing categories for dialogue systems (task-oriented and non-task-oriented), we named them as hidden-task-oriented dialogue systems.  ...  Though systems have a task, it is hidden from the users. Consequently, these may consider systems task-less.  ...  During the flow of the dialogue, storytelling dialogue systems change between task-oriented and non-task-oriented interactions.  ... 
dblp:conf/iui/TellolsTY20 fatcat:if32w3dy65fadduqyjh3bx76q4

Spatial querying for image retrieval

Joemon M. Jose, Jonathan Furner, David J. Harper
1998 Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '98  
A usercentred, task-oriented, comparative evaluation of Epic was undertaken in which two versions of the system--one set up to enable spatial queries only, the other allowing textual queries only-were  ...  Epic is an image retrieval system that implements a novel spatial-querying mechanism.  ...  Section 2, we identified the gradual emergence of a standardised framework for the evaluation of interactive, multimedia retrieval systems, and highlighted four methodological elements that are consistently  ... 
doi:10.1145/290941.291000 dblp:conf/sigir/JoseFH98 fatcat:sr27a2dxkrfq7jts27uujwsh4m

Report on the First International Workshop on the Evaluation on Collaborative Information Seeking and Retrieval (ECol'2015)

Laure Soulier, Lynda Tamine, Tetsuya Sakai, Leif Azzopardi, Jeremy Pickens
2016 SIGIR Forum  
Third, a discussion based on three seed research questions: (1) In what ways is collaborative search evaluation more challenging than individual interactive information retrieval (IIIR) evaluation?  ...  The discussion was intense and raised many points and issues, leading to the proposition that a new evaluation track focused on collaborative information retrieval/seeking tasks, would be worthwhile.  ...  Acknowledgments We thank ELIAS fundation for their grant #5907.  ... 
doi:10.1145/2964797.2964805 fatcat:g6gittiaavfnng3bl3lvc5ji5y

Overview of the Special Issue on Contextual Search and Recommendation

Paul N. Bennett, Diane Kelly, Ryen W. White, Yi Zhang
2015 ACM Transactions on Information Systems  
The interactions that occur during these complex tasks provide context that can be leveraged by search systems to support users' broader information-seeking activities.  ...  Furthermore, a variety of recent work and venues have noted that much information retrieval research on web search has focused on optimizing and evaluating single queries, even though a significant fraction  ...  Such complex tasks have additionally been the focus of a recent National Institute of Informatics sponsored workshop (Whole-Session Evaluation of Interactive Information Retrieval Systems [Belkin et al  ... 
doi:10.1145/2691351 fatcat:tgdtkoylfjf3xn3w2mcl3oqdcy

Overview of the CLEF 2017 Personalised Information Retrieval Pilot Lab (PIR-CLEF 2017) [chapter]

Gabriella Pasi, Gareth J. F. Jones, Stefania Marrara, Camilla Sanvitto, Debasis Ganguly, Procheta Sen
2017 Lecture Notes in Computer Science  
The Personalised Information Retrieval (PIR-CLEF) Lab workshop at CLEF 2017 is designed to provide a forum for the exploration of methodologies for the repeatable evaluation of personalised information  ...  for the evaluation of PIR and extension of the pilot Lab task.  ...  Introduction The objective of the PIR-CLEF Lab is to develop and demonstrate the effectiveness of a methodology for the repeatable evaluation of Personalised Information Retrieval (PIR).  ... 
doi:10.1007/978-3-319-65813-1_29 fatcat:ygtmjs5c5zbhvjj63vjvu6kiou
« Previous Showing results 1 — 15 out of 77,527 results