Log severity level classification: an approach for systems in production

TL;DR

This paper proposes an automated method for classifying log severity levels in production systems to reduce log noise, improve monitoring efficiency, and enhance fault diagnosis capabilities.

Contribution

It introduces a novel automated approach for classifying log severity levels, aiming to optimize log data management in production environments.

Findings

Automated severity classification reduces log noise.

Improved monitoring accuracy in production systems.

Enhanced fault diagnosis efficiency.

Abstract

Context: Logs are often the primary source of information for system developers and operations engineers to understand and diagnose the behavior of a software system in production. In many cases, logs are the only evidence available for fault investigation. Problem: However, the inappropriate choice of log severity level can impact the amount of log data generated and, consequently, quality. This storage overhead can impact the performance of log-based monitoring systems, as excess log data comes with increased aggregate noise, making it challenging to utilize what is actually important when trying to do diagnostics. Goal: This research aims to decrease the overheads of monitoring systems by processing the severity level of log data from systems in production. Approach: To achieve this goal, we intend to deepen the knowledge about the log severity levels and develop an automated approach to log severity level classification, demonstrating that reducing log severity level "noise" improves the monitoring of systems in production. Conclusion: We hope that the set of contributions from this work can improve the monitoring activities of software systems and contribute to the creation of knowledge that improves logging practices