FinEmbedDiff: A Cost-Effective Approach of Classifying Financial Documents with Vector Sampling using Multi-modal Embedding Models

TL;DR

FinEmbedDiff introduces a cost-effective multi-modal embedding sampling method for classifying complex financial documents, achieving high accuracy with reduced computational costs and strong generalization.

Contribution

It presents a novel vector sampling approach using pre-trained multi-modal embeddings for efficient financial document classification.

Findings

Achieves competitive accuracy with state-of-the-art methods

Reduces computational costs significantly

Demonstrates strong generalization on large datasets

Abstract

Accurate classification of multi-modal financial documents, containing text, tables, charts, and images, is crucial but challenging. Traditional text-based approaches often fail to capture the complex multi-modal nature of these documents. We propose FinEmbedDiff, a cost-effective vector sampling method that leverages pre-trained multi-modal embedding models to classify financial documents. Our approach generates multi-modal embedding vectors for documents, and compares new documents with pre-computed class embeddings using vector similarity measures. Evaluated on a large dataset, FinEmbedDiff achieves competitive classification accuracy compared to state-of-the-art baselines while significantly reducing computational costs. The method exhibits strong generalization capabilities, making it a practical and scalable solution for real-world financial applications.