LogBASA: Log Anomaly Detection Based on System Behavior Analysis and Global Semantic Awareness

• Published in: International journal of intelligent systems Vol. 2023; pp. 1 - 18
• Main Authors: Liao, Liping, Zhu, Ke, Luo, Jianzhen, Cai, Jun
• Format: Journal Article
• Language: English
• Published: New York Hindawi 20.09.2023; Hindawi Limited
• Subjects: Anomalies; Artificial intelligence; Coders; Data mining; Datasets; Deep learning; Effectiveness; Encoders-Decoders; False alarms; Fault diagnosis; Intelligent systems; Knowledge representation; Localization; Pattern recognition; Semantics; Sequences
• ISSN: 0884-8173; 1098-111X
• DOI: 10.1155/2023/3777826

System log anomaly detection is important for ensuring stable system operation and achieving rapid fault diagnosis. System log sequences include data on the execution paths and time stamps of system tasks in addition to a large amount of semantic information, which enhances the reliability and effectiveness of anomaly detection. At the same time, considering the correlation between system log sequences can effectively improve fault diagnosis efficiency. However, the existing system log anomaly detection methods mostly consider only the sequence patterns or semantic information on the logs, so their anomaly detection results show a high rate of missed and false alarms. To solve these problems, this paper proposed an unsupervised log anomaly detection model (LogBASA) based on the system behavior analysis and global semantic awareness, aiming to decrease the leakage rate and increase the log sequence anomaly detection accuracy. First, a system log knowledge graph was constructed based on massive, unstructured, and multilevel system log data to represent log sequence patterns, which facilitates subsequent anomaly detection and localization. Then, a self-attention encoder-decoder transformer model was developed for log spatiotemporal association analysis. This model combines semantic mapping and spatiotemporal features of log sequences to analyze system behavior and log semantics in multiple dimensions. Furthermore, a system log anomaly detection method that combines adaptive spatial boundary delineation and sequence reconstruction objective functions was proposed. This method uses special words to characterize the log sequence states, delineates anomaly boundaries automatically, and reconstructs log sequences through unsupervised training for anomaly detection. Finally, the proposed method was verified by numerous experiments on three real datasets. The results indicate that the proposed method can achieve an accuracy rate of 99.3%, 95.1%, and 97.2% on HDFS, BGL, and Thunderbird datasets, which proves the effectiveness and superiority of the LogBASA model.