Dissecting Global Search: A Simple Yet Effective Method to Boost Individual Discrimination Testing and Repair

• Published in: Proceedings / International Conference on Software Engineering pp. 1908 - 1920
• Main Authors: Quan, Lili, Li, Tianlin, Xie, Xiaofei, Chen, Zhenpeng, Chen, Sen, Jiang, Lingxiao, Li, Xiaohong
• Format: Conference Proceeding
• Language: English
• Published: IEEE 26.04.2025
• Subjects: Computational modeling; Decision making; Deep learning; DNNs; Fairness; Genetic algorithms; Individual Discrimination; Limiting; Maintenance engineering; Software engineering; Testing
• ISSN: 1558-1225
• DOI: 10.1109/ICSE55347.2025.00235

Deep Learning (DL) has achieved significant success in socially critical decision-making applications but often exhibits unfair behaviors, raising social concerns. Among these unfair behaviors, individual discrimination-examining inequalities between instance pairs with identical profiles differing only in sensitive attributes such as gender, race, and age-is extremely socially impactful. Existing methods have made significant and commendable efforts in testing individual discrimination before deployment. However, their efficiency and effectiveness remain limited, particularly when evaluating relatively fairer models. It remains unclear which phase of the existing testing framework (global or local) is the primary bottleneck limiting performance. Facing the above issues, we first identify that enhancing the global phase consistently improves overall testing effectiveness compared to enhancing the local phase. This motivates us to propose Genetic-Random Fairness Testing (GRFT), an effective and efficient method. In the global phase, we use a genetic algorithm to guide the search for more global discriminatory instances. In the local phase, we apply a light random search to explore the neighbors of these instances, avoiding time-consuming computations. Additionally, based on the fitness score, we also propose a straightforward yet effective repair approach. For a thorough evaluation, we conduct extensive experiments involving 6 testing methods, 5 datasets, 261 models (including 5 naively trained, 64 repaired, and 192 quantized for on-device deployment), and sixteen combinations of sensitive attributes, showing the superior performance of GRFT and our repair method.