FinRLlama: A Solution to LLM-Engineered Signals Challenge at FinRL Contest 2024

TL;DR

This paper introduces FinRLlama, a fine-tuning framework for large language models using reinforcement learning from market feedback, to generate more accurate trading signals for financial markets, winning the FinRL Challenge 2024.

Contribution

It presents a novel prompt-based fine-tuning approach for LLMs with market-specific data and reinforcement learning, tailored for financial trading signal generation.

Findings

Outperforms baseline methods in signal consistency

Achieves tighter trading outcomes

Wins the FinRL Challenge Task II

Abstract

In response to Task II of the FinRL Challenge at ACM ICAIF 2024, this study proposes a novel prompt framework for fine-tuning large language models (LLM) with Reinforcement Learning from Market Feedback (RLMF). Our framework incorporates market-specific features and short-term price dynamics to generate more precise trading signals. Traditional LLMs, while competent in sentiment analysis, lack contextual alignment for financial market applications. To bridge this gap, we fine-tune the LLaMA-3.2-3B-Instruct model using a custom RLMF prompt design that integrates historical market data and reward-based feedback. Our evaluation shows that this RLMF-tuned framework outperforms baseline methods in signal consistency and achieving tighter trading outcomes; awarded as winner of Task II. You can find the code for this project on GitHub.