FinRpt: Dataset, Evaluation System and LLM-based Multi-agent Framework for Equity Research Report Generation

TL;DR

This paper introduces the first dataset, evaluation system, and multi-agent framework for automated equity research report generation using large language models, addressing data scarcity and evaluation challenges.

Contribution

It formulates the novel ERR generation task, creates an open-source dataset and evaluation metrics, and proposes a multi-agent LLM-based framework for improved report generation.

Findings

High-quality ERR dataset created automatically

Effective evaluation metrics for ERR generation

Strong performance of the proposed framework

Abstract

While LLMs have shown great success in financial tasks like stock prediction and question answering, their application in fully automating Equity Research Report generation remains uncharted territory. In this paper, we formulate the Equity Research Report (ERR) Generation task for the first time. To address the data scarcity and the evaluation metrics absence, we present an open-source evaluation benchmark for ERR generation - FinRpt. We frame a Dataset Construction Pipeline that integrates 7 financial data types and produces a high-quality ERR dataset automatically, which could be used for model training and evaluation. We also introduce a comprehensive evaluation system including 11 metrics to assess the generated ERRs. Moreover, we propose a multi-agent framework specifically tailored to address this task, named FinRpt-Gen, and train several LLM-based agents on the proposed datasets using Supervised Fine-Tuning and Reinforcement Learning. Experimental results indicate the data quality and metrics effectiveness of the benchmark FinRpt and the strong performance of FinRpt-Gen, showcasing their potential to drive innovation in the ERR generation field. All code and datasets are publicly available.