Ticket-BERT: Labeling Incident Management Tickets with Language Models

TL;DR

Ticket-BERT is a specialized language model designed to accurately label complex incident management tickets, addressing unique challenges like frequent updates and diverse data sources, and it adapts quickly through active learning.

Contribution

The paper introduces Ticket-BERT, a robust language model tailored for incident ticket labeling, incorporating an active learning cycle for rapid adaptation to new data.

Findings

Ticket-BERT outperforms existing classifiers on Azure Cognitive Services.

It effectively handles ticket updates and diverse data sources.

Active learning enables quick fine-tuning with minimal annotations.

Abstract

An essential aspect of prioritizing incident tickets for resolution is efficiently labeling tickets with fine-grained categories. However, ticket data is often complex and poses several unique challenges for modern machine learning methods: (1) tickets are created and updated either by machines with pre-defined algorithms or by engineers with domain expertise that share different protocols, (2) tickets receive frequent revisions that update ticket status by modifying all or parts of ticket descriptions, and (3) ticket labeling is time-sensitive and requires knowledge updates and new labels per the rapid software and hardware improvement lifecycle. To handle these issues, we introduce Ticket- BERT which trains a simple yet robust language model for labeling tickets using our proposed ticket datasets. Experiments demonstrate the superiority of Ticket-BERT over baselines and state-of-the-art text classifiers on Azure Cognitive Services. We further encapsulate Ticket-BERT with an active learning cycle and deploy it on the Microsoft IcM system, which enables the model to quickly finetune on newly-collected tickets with a few annotations.