A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2007; you can also visit the original URL.
The file type is application/pdf.
Filters
Exploring Cost-Effective Approaches to Human Evaluation of Search Engine Relevance
[chapter]
2005
Lecture Notes in Computer Science
Preserved Fulltext
measure effectiveness relative to other search engines. ...
In this paper, we examine novel and less expensive methods for search engine evaluation that do not rely on document relevance judgments. ...
Acknowledgments: Thank you to Jan Pedersen at Yahoo! and the editorial team for hundreds of hours of work on judging search results. ...
doi:10.1007/978-3-540-31865-1_26
fatcat:ndvl2jjehvabpj2lqfte7xfv7y
Explaining User Performance in Information Retrieval: Challenges to IR Evaluation
[chapter]
2009
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
The paper makes three points of significance for IR research: (1) The Cranfield paradigm of IR evaluation seems to lose power when one looks at human instead of system performance. (2) Searchers using ...
Evaluation of output alone is insufficient to explain searcher behavior. ...
Acknowledgement This research was funded by the Academy of Finland grants #120996 and #124131. ...
doi:10.1007/978-3-642-04417-5_28
fatcat:d4ttunpujrblbairoqlohvuxna
Using the Crowd to Improve Search Result Ranking and the Search Experience
2016
ACM Transactions on Intelligent Systems and Technology
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
This paper explores how crowdsourcing can be used at query time to augment key stages of the search pipeline. We first explore the use of crowdsourcing to improve search result ranking. ...
While search tools that rely on human intervention will never be able to respond as quickly as current search engines do, recent research suggests that there are scenarios where a search engine could take ...
To make richer use of the crowd, we also explore several crowd-based approaches that change the existing search experience. ...
doi:10.1145/2897368
fatcat:symwsreokzcsvg6oko6lzi7h7a
Automated evaluation of search engine performance via implicit user feedback
2005
Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '05
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2006; you can also visit the original URL.
The file type is application/pdf.
Measuring the information retrieval effectiveness of Web search engines can be expensive if human relevance judgments are required to evaluate search results. ...
Web search engines can use human evaluation of search results without the expense of human evaluators. ...
engine performance by reducing the cost and time of evaluating retrieved results. ...
doi:10.1145/1076034.1076172
dblp:conf/sigir/SharmaJ05
fatcat:xnp3irh7vra5pfbysmgxvpkotu
Mining Implicit Relevance Feedback from User Behavior for Web Question Answering
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
In practice, this work has proved effective to substantially reduce the human labeling cost for the QA service in a global commercial search engine, especially for languages with low resources. ...
One principled idea is to mine implicit relevance feedback from user behavior recorded in search engine logs. ...
Our extensive experiments demonstrate the effectiveness of this approach in improving the performance of QA models and thus reducing the human labeling cost. ...
arXiv:2006.07581v1
fatcat:x24sblby6jhjxhrywdqmkdyt4q
Cost and benefit analysis of mediated enterprise search
2009
Proceedings of the 2009 joint international conference on Digital libraries - JCDL '09
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2011; you can also visit the original URL.
The file type is application/pdf.
evaluation also indicates the type of queries to be selected from a query log for evaluating an enterprise search engine. ...
We propose a method that evaluates the cost and benefit for a service provider of using a mediated search engine -in particular, where domain experts intervene on the ranking of the search results from ...
COST AND BENEFIT ANALYSIS In this section, we develop a cost and benefit analysis method to evaluate the cost effectiveness of the mediated enterprise search approach. ...
doi:10.1145/1555400.1555445
dblp:conf/jcdl/WuTTW09
fatcat:foofiza5c5g3nguovqoukrkpjy
Pre-trained Language Model based Ranking in Baidu Search
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
search engine system. ...
As the heart of a search engine, the ranking system plays a crucial role in satisfying users' information demands. ...
the efficiency and effectiveness of PLM-based ranking schema in the real-world search engine system. • Relevance-oriented Pre-training. ...
arXiv:2105.11108v3
fatcat:dbvj65ugovaani4hsiwtl6bcdi
Introducing natural language program analysis
2007
Proceedings of the 7th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering - PASTE '07
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
We then present an overview of our extraction process, program representation, and a set of tools we have developed using this natural language program analysis. ...
This research group presentation focuses on our work in extracting and utilizing natural language clues from source code to improve software maintenance tools. ...
Building effective software tools is important to reduce these high maintenance costs. ...
doi:10.1145/1251535.1251538
dblp:conf/paste/PollockVSHFM07
fatcat:gqerfkoixjfldgtkx2a72f4ss4
Crowdsourcing for search evaluation and social-algorithmic search
2012
Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval - SIGIR '12
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Today, Internet-based access to 24/7 online human crowds has led to a renaissance of research in human computation and the advent of crowdsourcing. ...
This tutorial introduces opportunities and challenges of human computation and crowdsourcing, particularly for search evaluation and developing hybrid search solutions that integrate human computation ...
Many Web search engines reportedly use large editorial staffs to judge the relevance of web pages for queries in an evaluation set, but his is expensive and has limited scalability. ...
doi:10.1145/2348283.2348530
dblp:conf/sigir/LeaseA12
fatcat:mjcbgbnlxrch7nb2xu2vgng57q
CapturingCollabportunities: A method to evaluate collaboration opportunities in information search using pseudocollaboration
2014
Journal of the Association for Information Science and Technology
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
We developed a technique called pseudocollaboration to evaluate the benefits and costs of collabportunities through simulations. ...
a largescale search engine with search sessions of thousands of searchers. ...
Acknowledgments The work described in this article was in part supported by the Institute of Museum and Library Services (IMLS) Early Career Development grant # RE-04-12-0105-12. ...
doi:10.1002/asi.23288
fatcat:6isney2ag5dlnpad2kenk3zgwq
Architecture-driven quality requirements prioritization
2012
2012 First IEEE International Workshop on the Twin Peaks of Requirements and Architecture (TwinPeaks)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
Our method uses automated design space exploration based on quantitative quality evaluation of software architecture models. ...
To decide for meaningful, feasible quality requirements and trade them off with functional requirements, tighter integration of software architecture evaluation and requirements prioritization is necessary ...
In the field of requirements engineering, numerous approaches to handle quality requirements have been suggested [21] , [22] . ...
doi:10.1109/twinpeaks.2012.6344554
dblp:conf/re/Koziolek12
fatcat:4d5j34f4gbfvferq4ryf7p63wm
Counterfactual Estimation and Optimization of Click Metrics for Search Engines
[article]
2014
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
The key challenge is the counterfactual nature: in the case of Web search, any change to a component of the search engine may result in a different search result page for the same query, but we normally ...
This approach effectively allows one to run (potentially infinitely) many A/B tests offline from search log, making it possible to estimate and optimize online metrics quickly and inexpensively. ...
INTRODUCTION The standard approach to evaluating ranking quality of a search engine is to evaluate its ranking results on a set of human-labeled examples and compute relevance metrics like mean average ...
arXiv:1403.1891v2
fatcat:fb6k5zsulbb23jl3tnd5is4pn4
Explore click models for search ranking
2010
Proceedings of the 19th ACM international conference on Information and knowledge management - CIKM '10
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2016; you can also visit the original URL.
The file type is application/pdf.
Yet, few works have been conducted to explore the use of click model to help web search ranking. ...
Recent advances in click model have positioned it as an effective approach to estimate document relevance based on user behavior in web search. ...
These click models have been considered as one of the most effective approaches to interpret user clicks and infer search relevance, and recent advances in click models have moved forward aggressively. ...
doi:10.1145/1871437.1871636
dblp:conf/cikm/WangCWZH10
fatcat:iuhl3fptqjbm5dydb5a5mdnynm
Efficient Test Collection Construction via Active Learning
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Other Versions
To create a new IR test collection at minimal cost, we must carefully select which documents merit human relevance judgments. ...
Results demonstrate the effectiveness of our approach, coupled with further analysis showing how varying relevance scarcity, within and across collections, impacts findings. ...
Even commercial search engines, despite their query logs, reportedly continue to depend on large teams of human Permission to make digital or hard copies of all or part of this work for personal or classroom ...
arXiv:1801.05605v2
fatcat:ssaz5gvat5h43njyf5difo7vju
Promoting transactions for A/E/C product information
2006
Automation in Construction
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
an existing search engine their capabilities in gathering additional product information. ...
Therefore, this researcher has developed a knowledge-supported approach to tackle the recognized issues to obtain an increased amount of product information from the available market, as a way to promote ...
The significance of efforts and resources needed to search for relevant product information is associated with the type of product being evaluated and the frequency with which a product is acquired. ...
doi:10.1016/j.autcon.2005.09.008
fatcat:hd3eod7ocvbfbh7cq2mygw2o44
« Previous
Showing results 1 — 15 out of 139,108 results