Improving Customer Service with Automatic Topic Detection in User Emails

TL;DR

This paper presents a natural language processing pipeline using BERTopic for automated email topic detection, significantly improving customer service efficiency at Telekom Srbija by accurately classifying emails in real-time.

Contribution

The study introduces a language-agnostic, modular NLP pipeline with high accuracy and speed for automated email topic detection, adaptable to low-resource languages.

Findings

Achieved a 0.041 second average processing time per email.

Attained a 0.96 weighted F1 score for topic classification.

System is deployed in production, enhancing customer service operations.

Abstract

This study introduces a novel natural language processing pipeline that enhances customer service efficiency at Telekom Srbija, a leading Serbian telecommunications company, through automated email topic detection and labeling. Central to the pipeline is BERTopic, a modular framework that allows unsupervised topic modeling. After a series of preprocessing and postprocessing steps, we assign one of 12 topics and several additional labels to incoming emails, allowing customer service to filter and access them through a custom-made application. While applied to Serbian, the methodology is conceptually language-agnostic and can be readily adapted to other languages, particularly those that are low-resourced and morphologically rich. The system performance was evaluated by assessing the speed and correctness of the automatically assigned topics, with a weighted average processing time of 0.041 seconds per email and a weighted average F1 score of 0.96. The system now operates in the company's production environment, streamlining customer service operations through automated email classification.