Cross‐version defect prediction using threshold‐based active learning

• Published in: Journal of software : evolution and process Vol. 36; no. 4
• Main Authors: Mei, Yuanqing, Liu, Xutong, Lu, Zeyu, Yang, Yibiao, Liu, Huihui, Zhou, Yuming
• Format: Journal Article
• Language: English
• Published: Chichester Wiley Subscription Services, Inc 01.04.2024
• Subjects: Active learning; cross‐version defect prediction (CVDP); Defects; Labels; Learning; median; Modules; Software; Software development; Subject specialists; threshold‐based active learning (TAL); Voting
• DOI: 10.1002/smr.2563
• ISSN: 2047-7473

Because defects in software modules (e.g., classes) might lead to product failure and financial loss, software defect prediction enables us to better understand and control software quality. Software development is a dynamic evolutionary process that may result in data distributions (e.g., defect characteristics) varying from version to version. In this case, effective cross‐version defect prediction (CVDP) is not easy to achieve. In this paper, we aim to investigate whether the defect prediction method of the threshold‐based active learning (TAL) can tackle the problem of the different data distribution between successive versions. Our TAL method includes two stages. At the active learning stage, a committee of investigated metrics is constructed to vote on the unlabeled modules of the current version. We pick up the unlabeled module with the median of voting scores to domain experts. The domain experts test and label the selected unlabeled module. Then, we merge the selected labeled module and the remaining modules with pseudo‐labels from the current version into the labeled modules of the prior version to form enhanced training data. Based on the training data, we derive the metric thresholds used for the next iteration. At the defect prediction stage, the iterations stop when a predefined threshold is reached. Finally, we use the cutoff threshold of voting scores, that is, 50%, to predict the defect‐prone of the remaining unlabeled modules. We evaluate the TAL method on 31 versions of 10 projects with three prevalent performance indicators. The results show that TAL outperforms the baseline methods, including three variations methods, two common supervised methods, and the state‐of‐the‐art method Hybrid Active Learning and Kernel PCA (HALKP). The results indicate that TAL can effectively address the different data distribution between successive versions. Furthermore, to keep the cost of extensive testing low in practice, selecting 5% of candidate modules from the current version is sufficient for TAL to achieve a good performance of defect prediction. We propose a threshold‐based active learning (TAL) approach to address the problem of underperformance of defect prediction due to the different data distributions between successive versions. TAL can actively select the unlabeled modules from the current version to domain experts for labeling and merge them into the prior version to mitigate the different data distributions. The results of our extensive experiments showed that TAL outperforms the baseline methods, including three variants, two common supervised methods, and the state‐of‐the‐art method Hybrid Active Learning and Kernel PCA (HALKP).