Financial News Summarization: Can extractive methods still offer a true alternative to LLMs?

TL;DR

This paper compares extractive summarization methods with large language models for financial news, highlighting that while LLMs produce more coherent summaries, extractive methods remain efficient and reliable for short, structured texts.

Contribution

The study provides a comprehensive evaluation of summarization techniques on financial news, emphasizing the trade-offs between LLMs and extractive methods in terms of quality and resource use.

Findings

LLMs generate more coherent and informative summaries.

Extractive methods are more efficient and less prone to hallucinations.

Fine-tuned LLMs achieve the best ROUGE scores, but with limited data reliability.

Abstract

Financial markets change rapidly due to news, economic shifts, and geopolitical events. Quick reactions are vital for investors to avoid losses or capture short-term gains. As a result, concise financial news summaries are critical for decision-making. With over 50,000 financial articles published daily, automation in summarization is necessary. This study evaluates a range of summarization methods, from simple extractive techniques to advanced large language models (LLMs), using the FinLLMs Challenge dataset. LLMs generated more coherent and informative summaries, but they are resource-intensive and prone to hallucinations, which can introduce significant errors into financial summaries. In contrast, extractive methods perform well on short, well-structured texts and offer a more efficient alternative for this type of article. The best ROUGE results come from fine-tuned LLM model like FT-Mistral-7B, although our data corpus has limited reliability, which calls for cautious interpretation.