Improving Segmentation for Technical Support Problems

TL;DR

This paper tackles the challenge of segmenting complex technical support questions by formulating it as a sequence labeling task, introducing a novel multi-model embedding approach, and demonstrating improved answer retrieval performance.

Contribution

It introduces a new segmentation method combining multiple pre-trained language models, enhancing accuracy over existing approaches and aiding answer retrieval in technical support.

Findings

Multi-model embeddings outperform single-model embeddings.

Segmentation improves answer retrieval accuracy.

State-of-the-art approaches are evaluated and compared.

Abstract

Technical support problems are often long and complex. They typically contain user descriptions of the problem, the setup, and steps for attempted resolution. Often they also contain various non-natural language text elements like outputs of commands, snippets of code, error messages or stack traces. These elements contain potentially crucial information for problem resolution. However, they cannot be correctly parsed by tools designed for natural language. In this paper, we address the problem of segmentation for technical support questions. We formulate the problem as a sequence labelling task, and study the performance of state of the art approaches. We compare this against an intuitive contextual sentence-level classification baseline, and a state of the art supervised text-segmentation approach. We also introduce a novel component of combining contextual embeddings from multiple language models pre-trained on different data sources, which achieves a marked improvement over using embeddings from a single pre-trained language model. Finally, we also demonstrate the usefulness of such segmentation with improvements on the downstream task of answer retrieval.