An automatically created novel bug dataset and its validation in bug prediction

• Published in: The Journal of systems and software Vol. 169; p. 110691
• Main Authors: Ferenc, Rudolf, Gyimesi, Péter, Gyimesi, Gábor, Tóth, Zoltán, Gyimóthy, Tibor
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01.11.2020
• Subjects: Bug dataset; Bug prediction; Code metrics; GitHub; Machine learning; Static code analysis; Bug prediction; Bug dataset; Code metrics; GitHub; Static code analysis; Machine learning
• ISSN: 0164-1212; 1873-1228
• DOI: 10.1016/j.jss.2020.110691

Bugs are inescapable during software development due to frequent code changes, tight deadlines, etc.; therefore, it is important to have tools to find these errors. One way of performing bug identification is to analyze the characteristics of buggy source code elements from the past and predict the present ones based on the same characteristics, using e.g. machine learning models. To support model building tasks, code elements and their characteristics are collected in so-called bug datasets which serve as the input for learning. We present the BugHunter Dataset: a novel kind of automatically constructed and freely available bug dataset containing code elements (files, classes, methods) with a wide set of code metrics and bug information. Other available bug datasets follow the traditional approach of gathering the characteristics of all source code elements (buggy and non-buggy) at only one or more pre-selected release versions of the code. Our approach, on the other hand, captures the buggy and the fixed states of the same source code elements from the narrowest timeframe we can identify for a bug’s presence, regardless of release versions. To show the usefulness of the new dataset, we built and evaluated bug prediction models and achieved F-measure values over 0.74. •Proposing a novel kind of bug dataset that captures before- and after-fix states.•Predicting faults at method level is more accurate than at class and file levels.•Using method level metrics for class level bug prediction performed the best.•The novel dataset moderates the uncertainty of labeling elements as buggy.