Automatically Categorizing Software Technologies

• Published in: IEEE transactions on software engineering Vol. 46; no. 1; pp. 20 - 32
• Main Authors: Nassif, Mathieu, Treude, Christoph, Robillard, Martin P.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.01.2020; IEEE Computer Society
• Subjects: Applications programs; Electronic publishing; Encyclopedias; information retrieval; Internet; natural language processing; Normalizing; Software; tagging; Taxonomy; Technology assessment; Wikipedia
• ISSN: 0098-5589; 1939-3520
• DOI: 10.1109/TSE.2018.2836450

Informal language and the absence of a standard taxonomy for software technologies make it difficult to reliably analyze technology trends on discussion forums and other on-line venues. We propose an automated approach called Witt for the categorization of software technologies (an expanded version of the hypernym discovery problem). Witt takes as input a phrase describing a software technology or concept and returns a general category that describes it (e.g., integrated development environment), along with attributes that further qualify it (commercial, php, etc.). By extension, the approach enables the dynamic creation of lists of all technologies of a given type (e.g., web application frameworks). Our approach relies on Stack Overflow and Wikipedia, and involves numerous original domain adaptations and a new solution to the problem of normalizing automatically-detected hypernyms. We compared Witt with six independent taxonomy tools and found that, when applied to software terms, Witt demonstrated better coverage than all evaluated alternative solutions, without a corresponding degradation in false positive rate.