Filters








14,188 Hits in 12.6 sec

A Comprehensive Empirical Study of Bias Mitigation Methods for Software Fairness [article]

Zhenpeng Chen, Jie M. Zhang, Federica Sarro, Mark Harman
2022 arXiv   pre-print
We present a large-scale, comprehensive empirical evaluation of 17 representative bias mitigation methods, evaluated with 12 Machine Learning (ML) performance metrics, 4 fairness metrics, and 24 types  ...  The empirical coverage is comprehensive, covering the largest numbers of bias mitigation methods, evaluation metrics, and fairness-performance trade-off measures compared to previous work on this important  ...  CONCLUSION This paper presents a large-scale empirical study evaluating 17 representative bias mitigation methods with 12 ML performance metrics, 4 fairness metrics, and 24 types of fairness-performance  ... 
arXiv:2207.03277v1 fatcat:yuz5p5lsofgq3njgo76pneoevy

Software Fairness: An Analysis and Survey [article]

Ezekiel Soremekun, Mike Papadakis, Maxime Cordy, Yves Le Traon
2022 arXiv   pre-print
However, the landscape of works addressing bias as a software engineering concern is unclear, i.e., techniques and studies that analyze the fairness properties of learning-based software.  ...  In this work, we provide a clear view of the state-of-the-art in software fairness analysis.  ...  [65] conducted a large scale empirical study to test the effectiveness of 12 widely-studied bias mitigation methods.  ... 
arXiv:2205.08809v1 fatcat:63whhiyjvvaida4kjvpsyd7gh4

Do the Machine Learning Models on a Crowd Sourced Platform Exhibit Bias? An Empirical Study on Model Fairness [article]

Sumon Biswas, Hridesh Rajan
2020 arXiv   pre-print
We have created a benchmark of 40 top-rated models from Kaggle used for 5 different tasks, and then using a comprehensive set of fairness metrics evaluated their fairness.  ...  Our study suggests future research directions to reduce the gap between theoretical fairness aware algorithms and the software engineering methods to leverage them in practice.  ...  All opinions are of the authors and do not reflect the view of sponsors.  ... 
arXiv:2005.12379v1 fatcat:y6q4vrprhfemjcpxo4igpsj47y

Fair Enough: Searching for Sufficient Measures of Fairness [article]

Suvodeep Majumder and Joymallya Chakraborty and Gina R. Bai and Kathryn T. Stolee and Tim Menzies
2022 arXiv   pre-print
Testing machine learning software for ethical bias has become a pressing current concern.  ...  In response, recent research has proposed a plethora of new fairness metrics, for example, the dozens of fairness metrics in the IBM AIF360 toolkit.  ...  There is much evidence of machine learning (ML) software showing biased behavior.  ... 
arXiv:2110.13029v2 fatcat:vya54lursjb57piyja7vxsiahe

xFAIR: Better Fairness via Model-based Rebalancing of Protected Attributes [article]

Kewen Peng, Joymallya Chakraborty, Tim Menzies
2022 arXiv   pre-print
Method: Here we propose xFAIR, a model-based extrapolation method, that is capable of both mitigating bias and explaining the cause.  ...  While those methods are effective in mitigating bias, few of them can provide explanations on what is the root cause of bias.  ...  ACKNOWLEDGEMENTS This work was partially funded by a research grant from the Laboratory for Analytical Sciences, North Carolina State University.  ... 
arXiv:2110.01109v2 fatcat:7nvzymlo4zdjjhk6kwzeduesfy

"Ignorance and Prejudice" in Software Fairness

Jie M. Zhang, Mark Harman
2021 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)  
This paper presents results from a comprehensive study that addresses this problem. We find that enlarging the feature set plays a significant role in fairness (with an average effect rate of 38%).  ...  Our results suggest a larger training data set has more unfairness than a smaller one when feature sets are insufficient; an important cautionary finding for practising software engineers.  ...  Empirical Studies on Software Fairness Chakraborty et al. [3] empirically studied the effectiveness and efficiency of existing fairness improvement methods.  ... 
doi:10.1109/icse43902.2021.00129 fatcat:oheycvbcvzghdecdohy4nkxflq

Algorithmic bias and the Value Sensitive Design approach

Judith Simon, Pak-Hang Wong, Gernot Rieder
2020 Internet Policy Review  
Relating these debates on values in design and algorithmic bias to research on cognitive biases, we conclude by stressing our collective duty to not only detect and counter biases in software systems,  ...  and the construction of novel technologies that account for specific desired values.  ...  For a comprehensive review of these critiques, see Davis and Nathan (2014) . develop a better understanding of bias in computer systems, not least because they considered biased systems to be "instruments  ... 
doi:10.14763/2020.4.1534 fatcat:oxpe34molncbvia43zdhmfz7ui

Astraea: Grammar-based Fairness Testing [article]

Ezekiel Soremekun and Sakshi Udeshi and Sudipta Chattopadhyay
2022 arXiv   pre-print
Using probabilistic grammars, ASTRAEA also provides fault diagnosis by isolating the cause of observed software bias. ASTRAEA's diagnoses facilitate the improvement of ML fairness.  ...  ASTRAEA was evaluated on 18 software systems that provide three major natural language processing (NLP) services. In our evaluation, ASTRAEA generated fairness violations with a rate of ~18%.  ...  This work was partially supported by the University of Luxembourg, Ezekiel Soremekun acknowledges the financial support of the Institute for Advanced Studies of the University of Luxembourg through an  ... 
arXiv:2010.02542v5 fatcat:n6ka7pbchrdczpnsgcjpomybfm

Fairness Testing: A Comprehensive Survey and Analysis of Trends [article]

Zhenpeng Chen, Jie M. Zhang, Max Hort, Federica Sarro, Mark Harman
2022 arXiv   pre-print
Research has focused on helping software engineers to detect fairness bugs automatically. This paper provides a comprehensive survey of existing research on fairness testing.  ...  Software systems are vulnerable to fairness bugs and frequently exhibit unfair behaviors, making software fairness an increasingly important concern for software engineers.  ...  ACKNOWLEDGMENTS Before submitting, we sent the paper to the authors of the collected papers, to check for accuracy and omission.  ... 
arXiv:2207.10223v2 fatcat:2k3zj2lr2fh7dhapetkm6irame

Seeing the Whole Elephant: Systematically Understanding and Uncovering Evaluation Biases in Automated Program Repair

Deheng Yang, Yan Lei, Xiaoguang Mao, Yuhua Qi, Xin Yi
2022 ACM Transactions on Software Engineering and Methodology  
Unfortunately, there is still no methodology to support a systematic comprehension and discovery of evaluation biases in APR, which impedes the mitigation of such biases and threatens the evaluation of  ...  As a result, we identify 17 investigated biases and uncover a new bias in the usage of patch validation strategies.  ...  This work is supported by the National Natural Science Foundation of China (No. 61872445) and the Major Key Project of PCL.  ... 
doi:10.1145/3561382 fatcat:dwa3xvn4rzatrer4pjfpokcvry

Algorithmic Fairness in Computational Medicine [article]

Jie Xu, Yunyu Xiao, Wendy Hui Wang, Yue Ning, Elizabeth A Shenkman, Jiang Bian, Fei Wang
2022 medRxiv   pre-print
Specifically, we overview the different types of algorithmic bias, fairness quantification metrics, and bias mitigation methods, and summarize popular software libraries and tools for bias evaluation and  ...  This paper presents a comprehensive review of algorithmic fairness in the context of computational medicine, which aims at improving medicine with computational approaches.  ...  Reweighing Reweighing is one of the pre-processing methods to mitigate the algorithm bias 49 . It is a simple but effective tool for minimizing bias.  ... 
doi:10.1101/2022.01.16.21267299 fatcat:26fp56upvfgabozuqowrrun3j4

Demographic Bias in Biometrics: A Survey on an Emerging Challenge

Pawel Drozdowski, Christian Rathgeb, Antitza Dantcheva, Naser Damer, Christoph Busch
2020 IEEE Transactions on Technology and Society  
The main contributions of this article are: 1) an overview of the topic of algorithmic bias in the context of biometrics; 2) a comprehensive survey of the existing literature on biometric bias estimation  ...  and mitigation; 3) a discussion of the pertinent technical and social matters; and 4) an outline of the remaining challenges and future work items, both from technological and social points of view.  ...  to empiric studies (especially in the case of bias mitigation, see Section III-D), stricter theoretical approaches need to be pursued in order to provably demonstrate the bias-mitigating properties of  ... 
doi:10.1109/tts.2020.2992344 fatcat:bi4arp3udnenjilwrlpnwfshfu

Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey [article]

Max Hort, Zhenpeng Chen, Jie M. Zhang, Federica Sarro, Mark Harman
2022 arXiv   pre-print
This paper provides a comprehensive survey of bias mitigation methods for achieving fairness in Machine Learning (ML) models.  ...  We collect a total of 341 publications concerning bias mitigation for ML classifiers.  ...  We would like to thank the members of the community who kindly provided comments and feedback on an earlier version of this paper.  ... 
arXiv:2207.07068v3 fatcat:6lge4ldht5fcvlw3tgbxbqevtq

Fair Preprocessing: Towards Understanding Compositional Fairness of Data Transformers in Machine Learning Pipeline [article]

Sumon Biswas, Hridesh Rajan
2021 arXiv   pre-print
For a machine learning task, it is a common practice to build a pipeline that includes an ordered set of data preprocessing stages followed by a classifier.  ...  In this paper, we introduced the causal method of fairness to reason about the fairness impact of data preprocessing stages in ML pipeline.  ...  Now, since undersampling method exhibits bias towards privileged group for XGB, we look for the transformer that is biased towards privileged group.  ... 
arXiv:2106.06054v1 fatcat:wohll3aefneilevsffgmbt3fya

Demographic Bias in Biometrics: A Survey on an Emerging Challenge [article]

P. Drozdowski, C. Rathgeb, A. Dantcheva, N. Damer, C. Busch
2020 arXiv   pre-print
The main contributions of this article are: (1) an overview of the topic of algorithmic bias in the context of biometrics, (2) a comprehensive survey of the existing literature on biometric bias estimation  ...  and mitigation, (3) a discussion of the pertinent technical and social matters, and (4) an outline of the remaining challenges and future work items, both from technological and social points of view.  ...  to empiric studies (especially in the case of bias mitigation, see subsection III-D), stricter theoretical approaches need to be pursued in order to provably demonstrate the bias-mitigating properties  ... 
arXiv:2003.02488v2 fatcat:s62lslhrcrfgvnqmp2d5yu7f7m
« Previous Showing results 1 — 15 out of 14,188 results