FinVis-GPT: A Multimodal Large Language Model for Financial Chart Analysis

TL;DR

FinVis-GPT is a multimodal large language model tailored for financial chart analysis, capable of interpreting charts, generating descriptions, answering questions, and predicting market trends, surpassing existing models.

Contribution

This paper introduces FinVis-GPT, the first multimodal LLM designed specifically for financial chart analysis, with a new dataset and superior performance in related tasks.

Findings

FinVis-GPT outperforms existing multimodal LLMs in financial tasks.

The model effectively generates descriptions and answers questions about financial charts.

A new dataset for financial chart analysis was created and will be publicly released.

Abstract

In this paper, we propose FinVis-GPT, a novel multimodal large language model (LLM) specifically designed for financial chart analysis. By leveraging the power of LLMs and incorporating instruction tuning and multimodal capabilities, FinVis-GPT is capable of interpreting financial charts and providing valuable analysis. To train FinVis-GPT, a financial task oriented dataset was generated for pre-training alignment and instruction tuning, comprising various types of financial charts and their corresponding descriptions. We evaluate the model performance via several case studies due to the time limit, and the promising results demonstrated that FinVis-GPT is superior in various financial chart related tasks, including generating descriptions, answering questions and predicting future market trends, surpassing existing state-of-the-art multimodal LLMs. The proposed FinVis-GPT serves as a pioneering effort in utilizing multimodal LLMs in the finance domain and our generated dataset will be release for public use in the near future to speedup related research.