Dynamic Adaptive Rank Space Exploration for Efficient Sentiment Analysis with Large Language Models

TL;DR

This paper introduces DARSE, a novel framework that dynamically explores and allocates ranks in large language models to enhance sentiment analysis accuracy while maintaining computational efficiency.

Contribution

The paper presents a new dynamic rank exploration framework that optimizes LLM fine-tuning for sentiment analysis, balancing accuracy and efficiency.

Findings

15.1% improvement in MSE

4.3% improvement in accuracy

Significant computational efficiency gains

Abstract

Sentiment analysis has become increasingly important for assessing public opinion and informing decision-making. Large language models (LLMs) have revolutionized this field by capturing nuanced language patterns. However, adapting LLMs to domain-specific sentiment analysis tasks remains challenging due to computational constraints and the need for optimal fine-tuning. To address these challenges, we propose a novel Dynamic Adaptive Rank Space Exploration (DARSE) framework for efficient and effective sentiment analysis using LLMs. DARSE consists of a coarse-grained greedy algorithm to identify the optimal rank range, a fine-grained exploration algorithm to refine rank selection, and a dynamic rank allocation method to determine the optimal rank combination for each LLM layer. Extensive experiments demonstrate that DARSE significantly improves sentiment analysis accuracy, achieving a 15.1% improvement in MSE and a 4.3% improvement in accuracy compared to previous work. Our framework strikes a balance between computational efficiency and model performance, making it a promising approach for sentiment analysis with LLMs.