Where were the repair ingredients for Defects4j bugs? Exploring the impact of repair ingredient retrieval on the performance of 24 program repair systems

• Published in: Empirical software engineering : an international journal Vol. 26; no. 6
• Main Authors: Yang, Deheng, Liu, Kui, Kim, Dongsun, Koyuncu, Anil, Kim, Kisub, Tian, Haoye, Lei, Yan, Mao, Xiaoguang, Klein, Jacques, Bissyandé, Tegawendé F.
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.11.2021
• Subjects: Compilers; Computer Science; Interpreters; Programming Languages; Software Engineering/Programming and Operating Systems; Donor Code; Code Change Action; Automated Program Repair; Fix Ingredient
• ISSN: 1382-3256; 1573-7616
• DOI: 10.1007/s10664-021-10003-7

A significant body of automated program repair research has built approaches under the redundancy assumption. Patches are then heuristically generated by leveraging repair ingredients (change actions and donor code) that are found in code bases (either the buggy program itself or big code). For example, common change actions (i.e., fix patterns) are frequently mined offline and serve as an important ingredient for many patch generation engines. Although the repetitiveness of code changes has been studied in general, the literature provides little insight into the relationship between the performance of the repair system and the source code base where the change actions were mined. Similarly, donor code is another important repair ingredient to concretize patches guided by abstract patterns. Yet, little attention has been paid to where such ingredients can actually be found. Through a large scale empirical study on the execution results of 24 repair systems evaluated on real-world bugs from Defects4J, we provide a comprehensive view on the distribution of repair ingredients that are relevant for these bugs. In particular, we show that (1) a half of bugs cannot be fixed simply because the relevant repair ingredient is not available in the search space of donor code; (2) bugs that are correctly fixed by literature tools are mostly addressed with shallow change actions; (3) programs with little history of changes can benefit from mining change actions in other programs; (4) parts of donor code to repair a given bug can be found separately at different search locations; (5) bug-triggering test cases are a rich source for donor code search.