Log Parsing: How Far Can ChatGPT Go?

Software logs play an essential role in ensuring the reliability and maintainability of large-scale software systems, as they are often the sole source of runtime information. Log parsing, which converts raw log messages into structured data, is an important initial step towards downstream log analy...

Full description

Saved in:
Bibliographic Details
Published in2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) pp. 1699 - 1704
Main Authors Le, Van-Hoang, Zhang, Hongyu
Format Conference Proceeding
LanguageEnglish
Published IEEE 11.09.2023
Subjects
Analytical models
Chatbots
ChatGPT
Data models
Large language model
Log analytics
Log parsing
Runtime
Software reliability
Software systems
Task analysis
Online AccessGet full text

Cover

More Information
Summary:Software logs play an essential role in ensuring the reliability and maintainability of large-scale software systems, as they are often the sole source of runtime information. Log parsing, which converts raw log messages into structured data, is an important initial step towards downstream log analytics. In recent studies, ChatGPT, the current cutting-edge large language model (LLM), has been widely applied to a wide range of software engineering tasks. However, its performance in automated log parsing remains unclear. In this paper, we evaluate ChatGPT's ability to undertake log parsing by addressing two research questions. (1) Can ChatGPT effectively parse logs? (2) How does ChatGPT perform with different prompting methods? Our results show that ChatGPT can achieve promising results for log parsing with appropriate prompts, especially with few-shot prompting. Based on our findings, we outline several challenges and opportunities for ChatGPT-based log parsing.
ISSN:2643-1572
DOI:10.1109/ASE56229.2023.00206