Filters








154,165 Hits in 3.7 sec

Adaptive Effort for Search Evaluation Metrics [chapter]

Jiepu Jiang, James Allan
2016 Lecture Notes in Computer Science  
We explain a wide range of search evaluation metrics as the ratio of users' gain to effort for interacting with a ranked list of results.  ...  Results show that such adaptive effort metrics better correlate with and predict user perceptions on search quality.  ...  Acknowledgment This work was supported in part by the Center for Intelligent Information Retrieval and in part by NSF grant #IIS-0910884.  ... 
doi:10.1007/978-3-319-30671-1_14 fatcat:qox4rsdyxbfebphbh3im5t3ppm

User behavior modeling for Web search evaluation

Fan Zhang, Yiqun Liu, Jiaxin Mao, Min Zhang, Shaoping Ma
2020 AI Open  
In this survey, we comprehensively review the development of user behavior modeling for Web search evaluation and related works of different model-based evaluation metrics.  ...  Recently, substantial studies have focused on proposing effective evaluation metrics that model user behavior to bring human factors in the loop of Web search evaluation.  ...  Adaptive effort metrics Similar to TBG (Smucker and Clarke, 2012a), Jiang and Allan (2016a) considered different effort for different documents and modified existing metrics to Adaptive Effort metrics.  ... 
doi:10.1016/j.aiopen.2021.02.003 fatcat:ir4hopyfy5asdoeddc3y64vk3a

Evaluating aggregated search pages

Ke Zhou, Ronan Cummins, Mounia Lalmas, Joemon M. Jose
2012 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval - SIGIR '12  
This paper proposes a general framework for evaluating the quality of aggregated search pages.  ...  We evaluate our approach by collecting annotated user preferences over a set of aggregated search pages for 56 topics and 12 verticals.  ...  In this way, all existing diversity-based IR metrics can be adapted to evaluate aggregated search.  ... 
doi:10.1145/2348283.2348302 dblp:conf/sigir/ZhouCLJ12 fatcat:b6yv5tsoongh5mnelokcfp3jpe

Maintainability defects detection and correction: a multi-objective approach

Ali Ouni, Marouane Kessentini, Houari Sahraoui, Mounir Boukadoum
2012 Automated Software Engineering : An International Journal  
For six open source projects, we succeeded in detecting the majority of known defects, and the proposed corrections fixed most of them with minimal effort.  ...  Then, we correct the detected defects while minimizing the correction effort.  ...  When evaluating the rules, actual metric values are mapped to truth values for the labels by means of membership functions.  ... 
doi:10.1007/s10515-011-0098-8 fatcat:jdgk3dcd2bbwxi5yymttijhlyy

Approaching the Dilemma between Plan and Value in Computer Aided Engineering of Production Machines

Denis Özdemir, Werner Herfs, Christian Brecher
2016 Procedia CIRP  
Modern mathematical optimization methods support the systematic search for a combination of system components and parameters with regards to the defined quality criteria.  ...  However, dynamic quality criteria that have a significant impact on the achievable accuracy and productivity cannot be evaluated with purely algebraic descriptions.  ...  Acknowledgement The authors would like to thank the German research Foundation DFG for the kind support within the project "Optimierung des Systementwurfs von Maschinen und Anlagen auf Basis komponentenorientierter  ... 
doi:10.1016/j.procir.2016.01.017 fatcat:ji44xjmzkrhh3pkbqkzt4ggjvq

Optimization Strategies in Design Space Exploration [chapter]

Jacopo Panerati, Donatella Sciuto, Giovanni Beltrame
2016 Handbook of Hardware/Software Codesign  
The algorithms are divided into four sub-classes and compared by means of several metrics: their setup effort, convergence rate, scalability, and performance of the optimization.  ...  The chapter describes and compares a selection of well-established multi-objective exploration algorithms for high-level design that appeared in recent scientific literature.  ...  On the contrary, if evaluations have moderate costs, one might want to trade-off a higher number of evaluations for a no-effort setup.  ... 
doi:10.1007/978-94-017-7358-4_7-1 fatcat:cnzulf533jcjpitbyl3xbgxx6e

Parallel Evolutionary Multi-Objective Optimization on Large, Heterogeneous Clusters: An Applications Perspective

Patrick M. Reed, Joshua B. Kollat, Matthew P. Ferringer, Timothy G. Thompson
2008 Journal of Aerospace Computing Information and Communication  
This study provides a cross-disciplinary collaborative effort to assess and adapt parallel multi-objective evolutionary algorithms for operational use in satellite constellation design using large dedicated  ...  A statistical, metric-based evaluation framework is used to demonstrate how time-continuation, asynchronous evolution, dynamic population sizing, and epsilon dominance archiving can be used to enhance  ...  This study provides a cross-disciplinary collaborative effort to assess and adapt parallel multi-objective evolutionary algorithms for operational use in satellite constellation design using large dedicated  ... 
doi:10.2514/1.36467 fatcat:uibu6uxqv5dhzbckgcyukhyk4e

EvalUMAP: Towards Comparative Evaluation in User Modeling, Adaptation and Personalization

Owen Conlan, Liadh Kelly, Kevin Koidl, Séamus Lawless, Athanasios Staikopoulos
2017 Conference and Labs of the Evaluation Forum  
There is currently no established or standardized means for the comparative evaluation of algorithms and systems developed by researchers in the User Modeling, Adaptation and Personalization (UMAP) space  ...  In this paper we present EvalUMAP, a new concerted drive towards the establishment of shared challenges for comparative evaluation within the UMAP community.  ...  Acknowledgements The ADAPT Centre for Digital Content Technology is funded under the SFI Research Centres Programme (Grant 13/RC/2106) and is co-funded under the European Regional Development Fund.  ... 
dblp:conf/clef/ConlanKKLS17 fatcat:vnlrn6rppjglvnoiudidq7jdmq

A systematic reuse process for automated acceptance tests: Construction and elementary evaluation

Mohsin Irshad, Kai Petersen
2021 e-Informatica Software Engineering Journal  
The techniques to support searching, assessing, adapting the reusable tests are proposed and evaluated.  ...  The process was evaluated for three constraints: performance expectancy, effort expectancy, and facilitating conditions.  ...  See 4.1.3 Selected approach to adapt AAT for reuse and to help in search Test Architect Manual A4: Search for reusable AATs Keywords from A1 and a search technique.  ... 
doi:10.37190/e-inf210107 fatcat:np6nfqbawfclzgwdaglwahoa2y

An Axiomatic Analysis of Diversity Evaluation Metrics

Enrique Amigó, Damiano Spina, Jorge Carrillo-de-Albornoz
2018 The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval - SIGIR '18  
Many evaluation metrics have been defined to evaluate the effectiveness ad-hoc retrieval and search result diversification systems.  ...  The analysis informed the definition of Rank-Biased Utility (RBU) -- an adaptation of the well-known Rank-Biased Precision metric -- that takes into account redundancy and the user effort associated to  ...  The authors wish to thank the reviewers for their valuable feedback.  ... 
doi:10.1145/3209978.3210024 dblp:conf/sigir/AmigoSA18 fatcat:4w6pugqbmjecnnq46fndzf7yzm

Convergence Acceleration Operator for Multiobjective Optimization

Salem F. Adra, Tony J. Dodd, Ian A. Griffin, Peter J. Fleming
2009 IEEE Transactions on Evolutionary Computation  
Associate in the Rolls-Royce University Technology Centre for Control and Systems Engineering, University of Sheffield, where he was responsible for research in advanced control laws for gas turbine engines  ...  The last two entries on the x-axis show the results when NSGA-II/CAO has adaptive step factors for the local search (A-rbf = adaptive CAO-RBF, A-Boxplots of the C-metric and DD-metric values achieved on  ...  Fig. 13 . 13 Boxplots of the C-metric and DD-metric values achieved on ZDT6-(fixed and adaptive local search).  ... 
doi:10.1109/tevc.2008.2011743 fatcat:ggw3xrb2nzhnrl5s25vigplfvq

Evaluating Search and Hyperlinking: An Example of the Design, Test, Refine Cycle for Metric Development

David Nicolas Racca, Gareth J. F. Jones
2015 MediaEval Benchmarking Initiative for Multimedia Evaluation  
We review evaluation metrics from the MediaEval Search and Hyperlinkiing task, illustrating the motivation behind metrics proposed for the task, and how reflection on results has led to iterative metric  ...  A considerable part of the effort in organising such a task must often be devoted to selecting, designing or refining a suitable evaluation metric.  ...  ACKNOWLEDGMENTS This work was supported by Science Foundation Ireland (Grant 12/CE/I2267) as part of the Centre for Global Intelligent Content CNGL II project at DCU.  ... 
dblp:conf/mediaeval/RaccaJ15 fatcat:g2f7nqtg6rfexl4yy63xyvl74m

Assessing the benefits of search-based approaches when designing self-adaptive systems: a controlled experiment

Sandro S Andrade, Raimundo J de A Macêdo
2015 Journal of Software Engineering Research and Development  
The experiment evaluated the design of self-adaptive systems using a search-based approach proposed by us, in contrast to the use of a non-automated approach based on architectural styles catalogs.  ...  Conclusion: This study contributes to reveal empirical evidence on the benefits of search-based approaches when designing self-adaptive systems architectures.  ...  Searching for effective self-adaptive systems architectures A number of efforts from the software engineering for SA systems community have addressed the issue of providing principled engineering approaches  ... 
doi:10.1186/s40411-015-0016-z fatcat:5l4efqesnfcopjd6rckcszzid4

On the reliability and intuitiveness of aggregated search metrics

Ke Zhou, Mounia Lalmas, Tetsuya Sakai, Ronan Cummins, Joemon M. Jose
2013 Proceedings of the 22nd ACM international conference on Conference on information & knowledge management - CIKM '13  
Although several aggregated search (AS) metrics have been proposed to evaluate AS result pages, their properties remain poorly understood.  ...  Aggregating search results from a variety of diverse verticals such as news, images, videos and Wikipedia into a single interface is a popular web search presentation paradigm.  ...  These metrics were proposed to evaluate the diversity of ranked lists over subtopics, and have been recently adapted to measure AS performance [27] . We discuss these adaptations next.  ... 
doi:10.1145/2505515.2505691 dblp:conf/cikm/ZhouLSCJ13 fatcat:utskfshfkzhonlh2jr5q5iuxbq

Adaptive relevance feedback for large-scale image retrieval

Nicolae Suditu, François Fleuret
2015 Multimedia tools and applications  
We have used the ImageNet dataset as it was released in 2010 for most of our evaluations in §4-6.  ...  We considered that this choice is more suitable for the generic evaluation of the adapted similarity metric. • Each user performed searching sessions corresponding to all combinations of systems and semantic  ...  This extension relies on an adaptive similarity metric that weights dynamically, at each iteration, the individual metrics of different modalities, depending on what the user is searching for.  ... 
doi:10.1007/s11042-015-2610-9 fatcat:v3gmot2r3rbmdnqc5lmdyu6w5y
« Previous Showing results 1 — 15 out of 154,165 results