Evaluating Fairness in Transaction Fraud Models: Fairness Metrics, Bias Audits, and Challenges
Ensuring fairness in transaction fraud detection models is vital due to the potential harms and legal implications of biased decision-making. Despite extensive research on algorithmic fairness, there is a notable gap in the study of bias in fraud detection models, mainly due to the field's uniq...
Saved in:
Main Authors | , , , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
06.09.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Ensuring fairness in transaction fraud detection models is vital due to the
potential harms and legal implications of biased decision-making. Despite
extensive research on algorithmic fairness, there is a notable gap in the study
of bias in fraud detection models, mainly due to the field's unique challenges.
These challenges include the need for fairness metrics that account for fraud
data's imbalanced nature and the tradeoff between fraud protection and service
quality. To address this gap, we present a comprehensive fairness evaluation of
transaction fraud models using public synthetic datasets, marking the first
algorithmic bias audit in this domain. Our findings reveal three critical
insights: (1) Certain fairness metrics expose significant bias only after
normalization, highlighting the impact of class imbalance. (2) Bias is
significant in both service quality-related parity metrics and fraud
protection-related parity metrics. (3) The fairness through unawareness
approach, which involved removing sensitive attributes such as gender, does not
improve bias mitigation within these datasets, likely due to the presence of
correlated proxies. We also discuss socio-technical fairness-related challenges
in transaction fraud models. These insights underscore the need for a nuanced
approach to fairness in fraud detection, balancing protection and service
quality, and moving beyond simple bias mitigation strategies. Future work must
focus on refining fairness metrics and developing methods tailored to the
unique complexities of the transaction fraud domain. |
---|---|
DOI: | 10.48550/arxiv.2409.04373 |