An Empirical Evaluation of Regression Testing Based on Fix-Cache Recommendations

• Published in: 2010 Third International Conference on Software Testing, Verification and Validation pp. 75 - 78
• Main Authors: Engström, Emelie, Runeson, Per, Wikstrand, Greger
• Format: Conference Proceeding
• Language: English
• Published: IEEE 2010
• Subjects: Costs; Embedded system; Fault diagnosis; Iterative methods; Joining processes; Logic testing; Monitoring; Software engineering; Software testing; System testing
• ISBN: 1424464358; 9781424464357
• ISSN: 2159-4848
• DOI: 10.1109/ICST.2010.40

Background: The fix-cache approach to regression test selection was proposed to identify the most fault-prone files and corresponding test cases through analysis of fixed defect reports. Aim: The study aims at evaluating the efficiency of this approach, compared to the previous regression test selection strategy in a major corporation, developing embedded systems. Method: We launched a post-hoc case study applying the fix-cache selection method during six iterations of development of a multi-million LOC product. The test case execution was monitored through the test management and defect reporting systems of the company. Results: From the observations, we conclude that the fix-cache method is more efficient in four iterations. The difference is statistically significant at α = 0.05. Conclusions: The new method is significantly more efficient in our case study. The study will be replicated in an environment with better control of the test execution.