Multimodal Financial Foundation Models (MFFMs): Progress, Prospects, and Challenges

TL;DR

This paper discusses the development and potential of Multimodal Financial Foundation Models (MFFMs) that integrate diverse financial data types to improve understanding and efficiency in financial services, highlighting ongoing research and future challenges.

Contribution

It introduces the concept of MFFMs, reviews current progress, prospects, and challenges, and presents ongoing research on FinAgents at Columbia University.

Findings

MFFMs can process diverse multimodal financial data.

They have potential to enhance financial decision-making.

Ongoing research aims to develop secure and effective FinAgents.

Abstract

Financial Large Language Models (FinLLMs), such as open FinGPT and proprietary BloombergGPT, have demonstrated great potential in select areas of financial services. Beyond this earlier language-centric approach, Multimodal Financial Foundation Models (MFFMs) can digest interleaved multimodal financial data, including fundamental data, market data, data analytics, macroeconomic, and alternative data (e.g., natural language, audio, images, and video). In this position paper, presented at the MFFM Workshop joined with ACM International Conference on AI in Finance (ICAIF) 2024, we describe the progress, prospects, and challenges of MFFMs. This paper also highlights ongoing research on FinAgents in the \textbf{SecureFinAI Lab}\footnote{\https://openfin.engineering.columbia.edu/} at Columbia University. We believe that MFFMs will enable a deeper understanding of the underlying complexity associated with numerous financial tasks and data, streamlining the operation of financial services and investment processes. Github Repo https://github.com/Open-Finance-Lab/Awesome-MFFMs/.