An empirical validation of FindBugs issues related to defects

• Published in: 15th Annual Conference on Evaluation & Assessment in Software Engineering (EASE 2011) pp. 144 - 153
• Main Authors: Vetro, A, Morisio, M, Torchiano, M
• Format: Conference Proceeding
• Language: English
• Published: Stevenage IET 2011; The Institution of Engineering & Technology
• Subjects: Diagnostic, testing, debugging and evaluating systems; Formal methods; Object-oriented programming; Programming support; Java; program debugging; coding phase; program verification; data flow analysis; bug finding tools; software tools; Java projects; source code verification; FindBugs issues
• ISBN: 9781849195096; 1849195099
• DOI: 10.1049/ic.2011.0018

Background: Effective use of bug finding tools promise to speed up the process of source code verification and to move a portion of discovered defects from testing to coding phase. However, many problems related to their usage, especially the large number of false positives, could easily hinder the potential benefits of such tools. Aims: Assess the percentage and type of issues of a popular bugfinding tool (FindBugs) that are actual defects. Method: We analyzed 301 Java Projects developed at a university with FindBugs, collecting the issues signalled on the source code. Afterwards, we checked the precision of issues with information on changes, we ranked and validated them using both manual inspection and validation with tests failures. Results: We observed that a limited set of issues have high precision and conversely we identified those issues characterized by low precision. We compared findings first with our previous experiment and then to related work: results are consistent with both of them. Conclusions: Since our and other empirical studies demonstrated that few issues are related to real defects with high precision, developers could enable only them (or prioritize), reducing the information overload of FindBugs and having the possibility to discover defects earlier. Furthermore, the technique presented in the paper can be adopted to other tools on a code base with tests to find issues with high precision that can be checked on code in production to find defects earlier.