Evaluating the understandability and user acceptance of Attack-Defense Trees: Original experiment and replication

• Published in: Information and software technology Vol. 178; p. 107624
• Main Authors: Broccia, Giovanna, ter Beek, Maurice H., Lluch Lafuente, Alberto, Spoletini, Paola, Fantechi, Alessandro, Ferrari, Alessio
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.02.2025
• Subjects: Attack-Defense Trees; Empirical user study; Internal replication; Method Evaluation Model; Security requirements; Understandability evaluation; Users acceptance; Understandability evaluation; Internal replication; Method Evaluation Model; Security requirements; Attack-Defense Trees; Users acceptance; Empirical user study
• ISSN: 0950-5849
• DOI: 10.1016/j.infsof.2024.107624

Attack-Defense Trees (ADTs) are a graphical notation used to model and evaluate security requirements. ADTs are popular because they facilitate communication among different stakeholders involved in system security evaluation and are formal enough to be verified using methods like model checking. The understandability and user-friendliness of ADTs are claimed as key factors in their success, but these aspects, along with user acceptance, have not been evaluated empirically. This paper presents an experiment with 25 subjects designed to assess the understandability and user acceptance of the ADT notation, along with an internal replication involving 49 subjects. The experiments adapt the Method Evaluation Model (MEM) to examine understandability variables (i.e., effectiveness and efficiency in using ADTs) and user acceptance variables (i.e., ease of use, usefulness, and intention to use). The MEM is also used to evaluate the relationships between these dimensions. In addition, a comparative analysis of the results of the two experiments is carried out. With some minor differences, the outcomes of the two experiments are aligned. The results demonstrate that ADTs are well understood by participants, with values of understandability variables significantly above established thresholds. They are also highly appreciated, particularly for their ease of use. The results also show that users who are more effective in using the notation tend to evaluate it better in terms of usefulness. These studies provide empirical evidence supporting both the understandability and perceived acceptance of ADTs, thus encouraging further adoption of the notation in industrial contexts, and development of supporting tools. •Experiment and replication to assess ADT notation’s understandability and acceptance.•Results: ADTs are understandable in terms of syntax, semantics, and practical use.•Results: ADTs are accepted, with high ease of use, usability, and intention to use.