Automatically Classifying Posts Into Question Categories on Stack Overflow

• Published in: 2018 IEEE/ACM 26th International Conference on Program Comprehension (ICPC) pp. 211 - 21110
• Main Authors: Beyer, Stefanie, Macho, Christian, Di Penta, Massimiliano, Pinzger, Martin
• Format: Conference Proceeding
• Language: English
• Published: ACM 01.05.2018
• Subjects: Machine Learning Algorithms; Question Categories; Stack Overflow
• ISSN: 2643-7171
• DOI: 10.1145/3196321.3196333

Software developers frequently solve development issues with the help of question and answer web forums, such as Stack Overflow (SO). While tags exist to support question searching and browsing, they are more related to technological aspects than to the question purposes. Tagging questions with their purpose can add a new dimension to the investigation of topics discussed in posts on SO. In this paper, we aim to automate such a classification of SO posts into seven question categories. As a first step, we have manually created a curated data set of 500 SO posts, classified into the seven categories. Using this data set, we apply machine learning algorithms (Random Forest and Support Vector Machines) to build a classification model for SO questions. We then experiment with 82 different configurations regarding the preprocessing of the text and representation of the input data. The results of the best performing models show that our models can classify posts into the correct question category with an average precision and recall of 0.88 and 0.87 when using Random Forest and the phrases indicating a question category as input data for the training. The obtained model can be used to aid developers in browsing so discussions or researchers in building recommenders based on SO.