Accelerating Bangla NLP Tasks with Automatic Mixed Precision: Resource-Efficient Training Preserving Model Efficacy

TL;DR

This paper demonstrates that automatic mixed precision training significantly accelerates Bangla NLP model training and reduces memory usage without compromising accuracy, making NLP development more accessible in resource-limited environments.

Contribution

The study is the first to evaluate AMP for Bangla NLP tasks, showing its effectiveness in resource-constrained settings with minimal performance loss.

Findings

Training speed increased by 44.5%.

Memory consumption reduced by 17.6%.

Model performance maintained within 0.3% of full precision.

Abstract

Training models for Natural Language Processing (NLP) requires substantial computational resources and time, posing significant challenges, especially for NLP development in Bangla, where access to high-end hardware is often limited. In this work, we explore automatic mixed precision (AMP) training as a means to improve computational efficiency without sacrificing model performance. By leveraging a dynamic mix of 16-bit and 32-bit floating-point computations, AMP lowers GPU memory requirements and speeds up training without degrading model performance. We evaluate AMP across four standard Bangla NLP tasks, namely sentiment analysis, named entity recognition, error classification, and question answering, using four transformer-based models: BanglaBERT, BanglishBERT, XLM-R, and mBERT. Our results demonstrate that AMP accelerates training by 44.5% and reduces memory consumption by 17.6%, while maintaining F-1 score within 99.7% of the full-precision baselines. This empirical study highlights AMP's potential to democratize access to state-of-the-art NLP capabilities in hardware-constrained settings by lowering computational barriers.