6,535 Hits in 8.6 sec

Assessing Viewpoint Diversity in Search Results Using Ranking Fairness Metrics [article]

Tim Draws, Nava Tintarev, Ujwal Gadiraju, Alessandro Bozzon, Benjamin Timmermans
2020 arXiv   pre-print
However, this viewpoint diversity is not trivial to assess. In this paper we use existing and novel ranking fairness metrics to evaluate viewpoint diversity in search result rankings.  ...  This paper lays out important ground work for future research to measure and assess viewpoint diversity in real search result rankings.  ...  In this paper, we investigate whether ranking fairness metrics can be used to assess viewpoint diversity in search results.  ... 
arXiv:2010.14531v1 fatcat:mivwonmrt5axpgvq6w5mx7tvzu

Understanding How Algorithmic and Cognitive Biases in Web Search Affect User Attitudes on Debated Topics

Tim Draws
2021 Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval  
One result of such biases is the search engine manipulation effect (SEME): when a list of search results relates to a debated topic (e.g., veganism) and promotes documents pertaining to a particular viewpoint  ...  (e.g., by ranking them higher), users tend to adopt this advantaged viewpoint [5] .  ...  The first contribution to RQ2 was demonstrating how to assess viewpoint-related ranking bias in search results using ranking fairness metrics for categorical viewpoint labels and evaluated which specific  ... 
doi:10.1145/3404835.3463273 fatcat:3b6kh43fwrforky7a2gebahy7e

Operationalizing Framing to Support Multiperspective Recommendations of Opinion Pieces

Mats Mulder, Oana Inel, Jasper Oosterman, Nava Tintarev
2021 Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency  
Our offline evaluation indicates that the proposed method is capable of enhancing the viewpoint diversity of recommendation lists according to a diversity metric from literature.  ...  While viewpoint diversity is often considered synonymous with source diversity in communication science domain, in this paper, we take a computational view.  ...  We would like to thank the team at Blendle for their support in this research, as well as the anonymous users that participated in our user study.  ... 
doi:10.1145/3442188.3445911 fatcat:holad4zstzfkbjqqlxyhsppll4

An empirical assessment of quality metrics for diversified similarity searching

Camila R. Lopes, Lúcio F. D. Santos, Daniel L. Jasbick, Daniel De Oliveira, Marcos Bedo
2021 Journal of Information and Data Management  
While several methods in information retrieval, data clustering, and similarity searching have tackled the problem of adding diversity into result sets, the experimental comparison of their performances  ...  We conclude diversity is better addressed by a set of measures rather than a single quality index and introduce the concept of Diversity Features Model (DFM), which combines the viewpoints of biased metrics  ...  , coined Diversity Features Model (DFM), which combines previous relevant metrics from data clustering and similarity-searching for assessing the quality of diversified similarity searching.  ... 
doi:10.5753/jidm.2021.1917 fatcat:syer3zrikbhsxcbtezjjhijpou

Evaluating diversified search results using per-intent graded relevance

Tetsuya Sakai, Ruihua Song
2011 Proceedings of the 34th international ACM SIGIR conference on Research and development in Information - SIGIR '11  
Search queries are often ambiguous and/or underspecified. To accomodate different user needs, search result diversification has received attention in the past few years.  ...  Our results show that diversified IR experiments with a given number of topics can be as reliable as traditional IR experiments with the same number of topics, provided that the right metrics are used.  ...  [8] proposed a metric called α-nDCG for evaluating diversified search results.  ... 
doi:10.1145/2009916.2010055 dblp:conf/sigir/SakaiS11 fatcat:rh6vfuva3fcmjbpgrujhuvmnf4

Comprehensive Viewpoint Representations for a Deeper Understanding of User Interactions With Debated Topics

Tim Draws, Oana Inel, Nava Tintarev, Christian Baden, Benjamin Timmermans
2022 ACM SIGIR Conference on Human Information Interaction and Retrieval  
For instance, a viewpoint's stance usually has a degree of strength (e.g., mild or strong), and, even if two viewpoints support or oppose something to the same degree, they may use different logics of  ...  By analyzing the resulting data set and conducting a user study, we further show that the two-dimensional viewpoint representation we propose allows for more meaningful analyses and diversification interventions  ...  Such labels can broadly categorize viewpoints while allowing for cheap computation of metrics and algorithms (e.g., related to ranking fairness [16, 60, 62] ).  ... 
doi:10.1145/3498366.3505812 fatcat:q2r7dxikpff63dynecwn2u2rvq

Developers' importance from the leader perspective

Guilherme Tangari, Marcelo Maia
2015 Proceedings of the 27th International Conference on Software Engineering and Knowledge Engineering  
Several companies use the amount of deliveries as a metric of performance evaluation of the developer.  ...  Metrics Productivity Past experiences Specialization (expert in some technology or tool) Generalization (diversity of skills) Solve complex problems Metrics The main behavior of the developer  ...  Survey 1) Goal-Question-Metric We used an approach called Goal Question Metric (GQM) [27] that helped us define our survey.  ... 
doi:10.18293/seke2015-56 dblp:conf/seke/TangariM15 fatcat:nr3ctklctvatteudhy2t5bz4qu

Evaluating epistemic uncertainty under incomplete assessments

Mark Baillie, Leif Azzopardi, Ian Ruthven
2008 Information Processing & Management  
In particular, we show through experimentation how the current practice in Information Retrieval evaluation of using a measurement depth larger than the pooling depth increases uncertainty during system  ...  This new methodology aims to identify potential uncertainty during system comparison that may result from incompleteness.  ...  This measures the concordance in the ranking of all systems in terms of Performance and Assessment metrics.  ... 
doi:10.1016/j.ipm.2007.04.002 fatcat:gabq7nsuyrhwjewo2eph2zi4rm

A Survey of Research on Fair Recommender Systems [article]

Yashar Deldjoo, Dietmar Jannach, Alejandro Bellogin, Alessandro Difonzo, Dario Zanzonelli
2022 arXiv   pre-print
Afterward, we provide a survey of how research in this area is currently operationalized, for example, in terms of the general research methodology, fairness metrics, and algorithmic approaches.  ...  a fair recommendation in the context of a given application.  ...  We discuss evaluation metrics next. Evaluation Metrics. We found that in offline experiments a wide range of computational metrics is used to assess the fairness of a set of recommendations.  ... 
arXiv:2205.11127v2 fatcat:qcq5iuwlevg2dh54i4on5jj4hi

Bias and unfairness in machine learning models: a systematic literature review [article]

Tiago Palma Pagano, Rafael Bessa Loureiro, Maira Matos Araujo, Fernanda Vitoria Nascimento Lisboa, Rodrigo Matos Peixoto, Guilherme Aragao de Sousa Guimaraes, Lucas Lisboa dos Santos, Gustavo Oliveira Ramos Cruz, Ewerton Lopes Silva de Oliveira, Marco Cruz, Ingrid Winkler, Erick Giovani Sperandio Nascimento
2022 arXiv   pre-print
The results show numerous bias and unfairness detection and mitigation approaches for ML technologies, with clearly defined metrics in the literature, and varied metrics can be highlighted.  ...  This study aims to examine existing knowledge on bias and unfairness in Machine Learning models, identifying mitigation methods, fairness metrics, and supporting tools.  ...  : Planning, Scoping, Searching, Assessing, Synthesizing.  ... 
arXiv:2202.08176v2 fatcat:m7rexaf26ra3zadl2dsqaqmjgq

Implementation of a Framework for Website Quality Evaluation: Himachal Pradesh University Website

Kalpana Devi, Aman Kumar Sharma
2016 Indian Journal of Science and Technology  
The prospective site quality measurements could be used as a site assessment system in order to assess functional sites and dispense superiority scores.  ...  In this paper the website superiority metrics and means to compute the website interface has been proposed.  ...  The functionality assessment result is depicted in Figure 3 . For the sake of fair evaluation of the degree of functionality in a webpage, every sub-attribute has to specify the weight.  ... 
doi:10.17485/ijst/2016/v9i40/100229 fatcat:fxbrimhxbvd73aksvy6olio2zu

What are you optimizing for? Aligning Recommender Systems with Human Values [article]

Jonathan Stray, Ivan Vendrov, Jeremy Nixon, Steven Adler, Dylan Hadfield-Menell
2021 arXiv   pre-print
We describe cases where real recommender systems were modified in the service of various human values such as diversity, fairness, well-being, time well spent, and factual accuracy.  ...  Instead, we look to AI alignment work for approaches that could learn complex values directly from stakeholders, and identify four major directions: useful measures of alignment, participatory design and  ...  Suites of metrics for "well-being" are used in public policy (O'Donnell et al., 2014 ) and many of those most applicable to AI systems have been collected into an IEEE standard for well-being assessment  ... 
arXiv:2107.10939v1 fatcat:e2mzj4l3lbg5ter5yo37pbki3m

Biobjective evolutionary and heuristic algorithms for intersection of geometric graphs

Rajeev Kumar, P.K. Singh, Bhargab B. Bhattacharya
2006 Proceedings of the 8th annual conference on Genetic and evolutionary computation - GECCO '06  
Search algorithms based on evolutionary multiobjective optimization (EMO) are then proposed to obtain diverse solutions in the feasible solution space.  ...  We first design simple heuristics to obtain the extreme points in the solution space, which however, could not produce diverse solutions.  ...  Kumar and Rockett [20] proposed use of rank-histograms to assess movement of solution-front towards convergence.  ... 
doi:10.1145/1143997.1144274 dblp:conf/gecco/KumarSB06 fatcat:q2a2l2q33vfvrn6esxy6bvncqm

Towards Algorithmic Transparency: A Diversity Perspective

Fausto Giunchiglia, Jahna Otterbacher, Styliani Kleanthous, Khuyagbaatar Batsuren, Veronika Bogin, Tsvi Kuflik, Avital Shulner Tal
2021 Zenodo  
Transparency is important for facilitating fairness management as well as explainability in algorithms; however, the concept of diversity, and its relationship to bias and transparency, has been largely  ...  As the role of algorithmic systems and processes increases in society, so does the risk of bias, which can result in discrimination against individuals and social groups.  ...  This was used as a benchmark of gender bias in the search engine results (presented to US-based users).  ... 
doi:10.5281/zenodo.5783298 fatcat:mjvzjbt5czboracqgfnqvfowx4

Popularity Bias in Recommendation: A Multi-stakeholder Perspective [article]

Himan Abdollahpouri
2020 arXiv   pre-print
In addition, I propose several algorithms each approaching the popularity bias mitigation from a different angle and compare their performances using several metrics with some other state-of-the-art approaches  ...  The effectiveness of these approaches, however, has not been assessed in multi-stakeholder environments.  ...  The algorithm, re-ranks an initial list of search results so it contains a fair proportion of web pages related to each of these two different aspects.  ... 
arXiv:2008.08551v1 fatcat:yiuamp6lcnc2bmhzkklakq6ui4
« Previous Showing results 1 — 15 out of 6,535 results