CryptoGPT: a 7B model rivaling GPT-4 in the task of analyzing and classifying real-time financial news

TL;DR

CryptoGPT is a 7-billion-parameter language model tailored for real-time financial news analysis in the cryptocurrency domain, achieving competitive performance with GPT-4 through strategic fine-tuning and semi-automatic annotation.

Contribution

The paper introduces CryptoGPT, a specialized LLM for financial news analysis that balances data privacy, annotation efficiency, model size, and analysis quality, outperforming similar-sized models.

Findings

CryptoGPT rivals GPT-4 in financial news analysis tasks.

Semi-automatic annotation improves model performance.

Fine-tuning with QLoRA enhances efficiency and accuracy.

Abstract

CryptoGPT: a 7B model competing with GPT-4 in a specific task -- The Impact of Automatic Annotation and Strategic Fine-Tuning via QLoRAIn this article, we present a method aimed at refining a dedicated LLM of reasonable quality with limited resources in an industrial setting via CryptoGPT. It is an LLM designed for financial news analysis for the cryptocurrency market in real-time. This project was launched in an industrial context. This model allows not only for the classification of financial information but also for providing comprehensive analysis. We refined different LLMs of the same size such as Mistral-7B and LLama-7B using semi-automatic annotation and compared them with various LLMs such as GPT-3.5 and GPT-4. Our goal is to find a balance among several needs: 1. Protecting data (by avoiding their transfer to external servers), 2. Limiting annotation cost and time, 3. Controlling the model's size (to manage deployment costs), and 4. Maintaining better analysis quality.