MS-guided many-objective evolutionary optimisation for test suite minimisation

• Published in: IET software Vol. 12; no. 6; pp. 547 - 554
• Main Authors: Zheng, Wei, Wu, Xiaoxue, Cao, Shichao, Lin, Jun
• Format: Journal Article
• Language: English
• Published: The Institution of Engineering and Technology 01.12.2018
• Subjects: genetic algorithms; many-objective optimisation problem; minimisation; MS-guided many-objective evolutionary optimisation; MS-guided many-objective optimisation approach; mutation score-guided many-objective optimisation approach; nondominated sorting genetic algorithm II; NSGA-II; obsolete test cases; optimal test suite; Pareto optimisation; program space; program testing; redundant test cases; Research Article; software-artefact infrastructure repository; standard code coverage criteria; statistical analysis; test cost; test suite minimisation process; Pareto optimisation; NSGA-II; program testing; mutation score-guided many-objective optimisation approach; optimal test suite; nondominated sorting genetic algorithm II; genetic algorithms; MS-guided many-objective evolutionary optimisation; many-objective optimisation problem; program space; test cost; test suite minimisation process; standard code coverage criteria; software-artefact infrastructure repository; statistical analysis; minimisation; redundant test cases; obsolete test cases; MS-guided many-objective optimisation approach
• ISSN: 1751-8806; 1751-8814; 1751-8814
• DOI: 10.1049/iet-sen.2018.5133

Test suite minimisation is a process that seeks to identify and then eliminate the obsolete or redundant test cases from the test suite. It is a trade-off between cost and other value criteria and is appropriate to be described as a many-objective optimisation problem. This study introduces a mutation score (MS)-guided many-objective optimisation approach, which prioritises the fault detection ability of test cases and takes MS, cost and three standard code coverage criteria as objectives for the test suite minimisation process. They use six classical evolutionary many-objective optimisation algorithms to identify efficient test suite, and select three small programs from the Software-Artefact Infrastructure Repository (SIR) and two larger program space and gzip for experimental evaluation as well as statistical analysis. The experiment results of the three small programs show non-dominated sorting genetic algorithm II (NSGA-II) with tuning was the most effective approach. However, MOEA/D-PBI and MOEA/D-WS outperform NSGA-II in the cases of two large programs. On the other hand, the test cost of the optimal test suite obtained by their proposed MS-guided many-objective optimisation approach is much lower than the one without it in most situation for both small programs and large programs.