Fast and Accurate FSA System Using ELBERT: An Efficient and Lightweight BERT

TL;DR

This paper introduces ELBERT, a lightweight BERT variant with a confidence-window-based early exit mechanism, enabling a fast and accurate financial sentiment analysis system that significantly outperforms existing methods in speed and maintains high accuracy.

Contribution

The paper presents ELBERT and a novel acceleration method with an effective early exit mechanism, improving speed and accuracy for BERT-based NLP tasks, especially in financial sentiment analysis.

Findings

Achieves nearly 40x speedup in text processing.

Outperforms FastBERT in speed while maintaining high accuracy.

Provides a practical solution for real-time financial sentiment analysis.

Abstract

With the development of deep learning and Transformer-based pre-trained models like BERT, the accuracy of many NLP tasks has been dramatically improved. However, the large number of parameters and computations also pose challenges for their deployment. For instance, using BERT can improve the predictions in the financial sentiment analysis (FSA) task but slow it down, where speed and accuracy are equally important in terms of profits. To address these issues, we first propose an efficient and lightweight BERT (ELBERT) along with a novel confidence-window-based (CWB) early exit mechanism. Based on ELBERT, an innovative method to accelerate text processing on the GPU platform is developed, solving the difficult problem of making the early exit mechanism work more effectively with a large input batch size. Afterward, a fast and high-accuracy FSA system is built. Experimental results show that the proposed CWB early exit mechanism achieves significantly higher accuracy than existing early exit methods on BERT under the same computation cost. By using this acceleration method, our FSA system can boost the processing speed by nearly 40 times to over 1000 texts per second with sufficient accuracy, which is nearly twice as fast as FastBERT, thus providing a more powerful text processing capability for modern trading systems.