Scalable and Efficient Large-Scale Log Analysis with LLMs: An IT Software Support Case Study

TL;DR

This paper presents a scalable log analysis tool using Large Language Models (LLMs) for IT support, demonstrating significant efficiency gains and cost savings in processing large log volumes across multiple software products.

Contribution

The paper introduces a novel approach for efficiently deploying LLMs on CPUs for large-scale log analysis, with real-world deployment insights and substantial performance improvements.

Findings

Processed over 2000 tickets across 70 products

Saved 300+ man hours per month

Reduced manpower costs by approximately $15,444 monthly

Abstract

IT environments typically have logging mechanisms to monitor system health and detect issues. However, the huge volume of generated logs makes manual inspection impractical, highlighting the importance of automated log analysis in IT Software Support. In this paper, we propose a log analytics tool that leverages Large Language Models (LLMs) for log data processing and issue diagnosis, enabling the generation of automated insights and summaries. We further present a novel approach for efficiently running LLMs on CPUs to process massive log volumes in minimal time without compromising output quality. We share the insights and lessons learned from deployment of the tool - in production since March 2024 - scaled across 70 software products, processing over 2000 tickets for issue diagnosis, achieving a time savings of 300+ man hours and an estimated $15,444 per month in manpower costs compared to the traditional log analysis practices.