Multi-class Text Classification using BERT-based Active Learning

TL;DR

This paper investigates the use of active learning strategies with BERT to efficiently classify short, incoherent, and code-mixed customer transaction descriptions in the pickup and delivery industry, reducing labeling costs.

Contribution

It introduces active learning approaches tailored for BERT to improve multi-class text classification on challenging, real-world customer data with minimal manual labeling.

Findings

Active learning improves labeling efficiency for BERT models.

BERT-based active learning outperforms random sampling in accuracy.

Effective in classifying short, incoherent, and code-mixed texts.

Abstract

Text Classification finds interesting applications in the pickup and delivery services industry where customers require one or more items to be picked up from a location and delivered to a certain destination. Classifying these customer transactions into multiple categories helps understand the market needs for different customer segments. Each transaction is accompanied by a text description provided by the customer to describe the products being picked up and delivered which can be used to classify the transaction. BERT-based models have proven to perform well in Natural Language Understanding. However, the product descriptions provided by the customers tend to be short, incoherent and code-mixed (Hindi-English) text which demands fine-tuning of such models with manually labelled data to achieve high accuracy. Collecting this labelled data can prove to be expensive. In this paper, we explore Active Learning strategies to label transaction descriptions cost effectively while using BERT to train a transaction classification model. On TREC-6, AG's News Corpus and an internal dataset, we benchmark the performance of BERT across different Active Learning strategies in Multi-Class Text Classification.