Diversity of graph models and graph generators in mutation testing

• Published in: International journal on software tools for technology transfer Vol. 22; no. 1; pp. 57 - 78
• Main Authors: Semeráth, Oszkár, Farkas, Rebeka, Bergmann, Gábor, Varró, Dániel
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.02.2020; Springer Nature B.V
• Subjects: Computer Science; Cyber-physical systems; Fase 2018; Modelling; Mutation; Safety critical; Software Engineering; Software Engineering/Programming and Operating Systems; Software testing; Theory of Computation; Model generators; Model diversity; Graph diversity metrics; Shape analysis; Mutation testing
• ISSN: 1433-2779; 1433-2787
• DOI: 10.1007/s10009-019-00530-6

When custom modeling tools are used for designing complex safety-critical systems (e.g., critical cyber-physical systems), the tools themselves need to be validated by systematic testing to prevent tool-specific bugs reaching the system. Testing of such modeling tools relies upon an automatically generated set of models as a test suite. While many software testing practices recommend that this test suite should be diverse, model diversity has not been studied systematically for graph models. In the paper, we propose different diversity metrics for models by generalizing and exploiting neighborhood and predicate shapes as abstraction. We evaluate such shape-based diversity metrics using various distance functions in the context of mutation testing of graph constraints and access policies for two separate industrial DSLs. Furthermore, we evaluate the quality (i.e., bug detection capability) of different (random and consistent) model generation techniques for mutation testing purposes.