Improving fault localization for Simulink models using search-based testing and prediction models

• Published in: 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER) pp. 359 - 370
• Main Authors: Bing Liu, Lucia, Nejati, Shiva, Briand, Lionel C.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.02.2017
• Subjects: Adaptation models; Computational modeling; Debugging; Fault localization; Predictive models; Ranking (statistics); search-based testing; Simulink models; Software packages; supervised learning; test suite diversity; Testing
• DOI: 10.1109/SANER.2017.7884636

One promising way to improve the accuracy of fault localization based on statistical debugging is to increase diversity among test cases in the underlying test suite. In many practical situations, adding test cases is not a cost-free option because test oracles are developed manually or running test cases is expensive. Hence, we require to have test suites that are both diverse and small to improve debugging. In this paper, we focus on improving fault localization of Simulink models by generating test cases. We identify three test objectives that aim to increase test suite diversity. We use these objectives in a search-based algorithm to generate diversified but small test suites. To further minimize test suite sizes, we develop a prediction model to stop test generation when adding test cases is unlikely to improve fault localization. We evaluate our approach using three industrial subjects. Our results show (1) the three selected test objectives are able to significantly improve the accuracy of fault localization for small test suite sizes, and (2) our prediction model is able to maintain almost the same fault localization accuracy while reducing the average number of newly generated test cases by more than half.