An Interactive Web-Based Toolset for Knowledge Discovery from Short Text Log Data

• Published in: Advanced Data Mining and Applications Vol. 10604; pp. 853 - 858
• Main Authors: Stewart, Michael, Liu, Wei, Cardell-Oliver, Rachell, Griffin, Mark
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2017; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Knowledge discovery; Unstructured data mining; Visualisation
• ISBN: 9783319691787; 3319691783
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-69179-4_61

Many companies maintain human-written logs to capture data on events such as workplace incidents and equipment failures. However, the sheer volume and unstructured nature of this data prevent it from being utilised for knowledge acquisition. Our web-based prototype software system provides a cohesive computational methodology for analysing and visualising log data that requires minimal human involvement. It features an interface to support customisable, modularised log data processing and knowledge discovery. This enables owners of event-based datasets containing short textual descriptions, such as occupational health & safety officers and machine operators, to identify latent knowledge not previously acquirable without significant time and effort. The software system comprises five distinct stages, corresponding to standard data mining milestones: exploratory analysis, data warehousing, association rule mining, entity clustering, and predictive analysis. To the best of our knowledge, it is the first dedicated system to computationally analyse short text log data and provides a powerful interface that visualises the analytical results and supports human interaction.