Financial Text Classification Based On rLoRA Finetuning On Qwen3-8B model

TL;DR

This paper evaluates the large language model Qwen3-8B for financial text classification tasks, demonstrating its superior accuracy and efficiency over classical and large-scale models, with potential for real-time financial NLP applications.

Contribution

The study introduces a novel fine-tuning approach for Qwen3-8B using Noisy Embedding Instruction Finetuning and optimization techniques, enhancing its performance and efficiency in financial NLP tasks.

Findings

Qwen3-8B outperforms classical transformer models and large-scale models in accuracy.

The model requires fewer training epochs to achieve better results.

Fine-tuning methods improve robustness and efficiency in financial text classification.

Abstract

Financial text classification has increasingly become an important aspect in quantitative trading systems and related tasks, such as financial sentiment analysis and the classification of financial news. In this paper, we assess the performance of the large language model Qwen3-8B on both tasks. Qwen3-8B is a state-of-the-art model that exhibits strong instruction-following and multilingual capabilities, and is distinct from standard models, primarily because it is specifically optimized for efficient fine tuning and high performance on reasoning-based benchmarks, making it suitable for financial applications. To adapt this model, we apply Noisy Embedding Instruction Finetuning and based on our previous work, this method increases robustness by injecting controlled noise into the embedding layers during supervised adaptation. We improve efficiency further with Rank-stabilized Low-Rank Adaptation low-rank optimization approach, and FlashAttention, which allow for faster training with lower GPU memory. For both tasks, we benchmark Qwen3-8B against standard classical transformer models, such as T5, BERT, and RoBERTa, and large models at scale, such as LLaMA1-7B, LLaMA2-7B, and Baichuan2-7B. The findings reveal that Qwen3-8B consistently surpasses these baselines by obtaining better classification accuracy and needing fewer training epochs. The synergy of instruction-based fine-tuning and memory-efficient optimization methods suggests Qwen3-8B can potentially serve as a scalable, economical option for real-time financial NLP applications. Qwen3-8B provides a very promising base for advancing dynamic quantitative trading systems in the future.