Are They All Good? Studying Practitioners' Expectations on the Readability of Log Messages

TL;DR

This study investigates practitioners' expectations on log message readability, identifies key aspects influencing readability, and demonstrates that machine learning models can effectively classify log message quality to improve logging practices.

Contribution

It provides new insights into readability standards for log messages, introduces practical guidelines, and explores automated classification using machine learning.

Findings

38.1% of log messages have inadequate readability

Deep learning models achieve over 80% accuracy in classification

Practitioners value structure, information, and wording in log messages

Abstract

Developers write logging statements to generate logs that provide run-time information for various tasks. The readability of log messages in the logging statements (i.e., the descriptive text) is rather crucial to the value of the generated logs. Immature log messages may slow down or even obstruct the process of log analysis. Despite the importance of log messages, there is still a lack of standards on what constitutes good readability in log messages and how to write them. In this paper, we conduct a series of interviews with 17 industrial practitioners to investigate their expectations on the readability of log messages. Through the interviews, we derive three aspects related to the readability of log messages, including Structure, Information, and Wording, along with several specific practices to improve each aspect. We validate our findings through a series of online questionnaire surveys and receive positive feedback from the participants. We then manually investigate the readability of log messages in large-scale open source systems and find that a large portion (38.1%) of the log messages have inadequate readability. Motivated by such observation, we further explore the potential of automatically classifying the readability of log messages using deep learning and machine learning models. We find that both deep learning and machine learning models can effectively classify the readability of log messages with a balanced accuracy above 80.0% on average. Our study provides comprehensive guidelines for composing log messages to further improve practitioners' logging practices.