TrioXpert: An Automated Incident Management Framework for Microservice System

TL;DR

TrioXpert is an innovative framework that leverages multimodal data and large language models to improve incident management tasks in microservice systems, offering enhanced accuracy and interpretability.

Contribution

It introduces a multimodal, end-to-end incident management framework with a collaborative reasoning mechanism using LLMs for multiple tasks.

Findings

Significant performance improvements in anomaly detection, failure triage, and root cause localization.

Effective deployment in real-world production environment.

Enhanced interpretability with clear reasoning evidence.

Abstract

Automated incident management plays a pivotal role in large-scale microservice systems. However, many existing methods rely solely on single-modal data (e.g., metrics, logs, and traces) and struggle to simultaneously address multiple downstream tasks, including anomaly detection (AD), failure triage (FT), and root cause localization (RCL). Moreover, the lack of clear reasoning evidence in current techniques often leads to insufficient interpretability. To address these limitations, we propose TrioXpert, an end-to-end incident management framework capable of fully leveraging multimodal data. TrioXpert designs three independent data processing pipelines based on the inherent characteristics of different modalities, comprehensively characterizing the operational status of microservice systems from both numerical and textual dimensions. It employs a collaborative reasoning mechanism using large language models (LLMs) to simultaneously handle multiple tasks while providing clear reasoning evidence to ensure strong interpretability. We conducted extensive evaluations on two microservice system datasets, and the experimental results demonstrate that TrioXpert achieves outstanding performance in AD (improving by 4.7% to 57.7%), FT (improving by 2.1% to 40.6%), and RCL (improving by 1.6% to 163.1%) tasks. TrioXpert has also been deployed in Lenovo's production environment, demonstrating substantial gains in diagnostic efficiency and accuracy.