Picking Pearl From Seabed: Extracting Artefacts from Noisy Issue Triaging Collaborative Conversations for Hybrid Cloud Services

TL;DR

This paper introduces a hybrid approach combining unsupervised and supervised learning to extract relevant artefacts from noisy, unlabelled collaborative conversations in cloud service issue triaging, improving accuracy with minimal human input.

Contribution

It presents a novel ensemble method that leverages domain knowledge and minimal labelled data to effectively extract artefacts from noisy conversations, enhancing issue triaging processes.

Findings

The ensemble model outperforms individual models in artefact extraction accuracy.

Minimal labelled data combined with domain knowledge improves model performance.

Experimental results demonstrate the effectiveness of the proposed approach.

Abstract

Site Reliability Engineers (SREs) play a key role in issue identification and resolution. After an issue is reported, SREs come together in a virtual room (collaboration platform) to triage the issue. While doing so, they leave behind a wealth of information which can be used later for triaging similar issues. However, usability of the conversations offer challenges due to them being i) noisy and ii) unlabelled. This paper presents a novel approach for issue artefact extraction from the noisy conversations with minimal labelled data. We propose a combination of unsupervised and supervised model with minimum human intervention that leverages domain knowledge to predict artefacts for a small amount of conversation data and use that for fine-tuning an already pretrained language model for artefact prediction on a large amount of conversation data. Experimental results on our dataset show that the proposed ensemble of unsupervised and supervised model is better than using either one of them individually.