Exploiting Network Structures to Improve Semantic Representation for the Financial Domain

TL;DR

This paper introduces a method combining transformer-based contextual embeddings with knowledge graph embeddings to enhance semantic representations in the financial domain, improving accuracy in semantic similarity tasks.

Contribution

The paper proposes a novel approach that integrates language model embeddings with knowledge graph structures and a voting mechanism for better financial entity representation.

Findings

Knowledge graph embeddings improve model performance.

Voting function enhances final inference accuracy.

Combined models outperform single embedding approaches.

Abstract

This paper presents the participation of the MiniTrue team in the FinSim-3 shared task on learning semantic similarities for the financial domain in English language. Our approach combines contextual embeddings learned by transformer-based language models with network structures embeddings extracted on external knowledge sources, to create more meaningful representations of financial domain entities and terms. For this, two BERT based language models and a knowledge graph embedding model are used. Besides, we propose a voting function to joint three basic models for the final inference. Experimental results show that the model with the knowledge graph embeddings has achieved a superior result than these models with only contextual embeddings. Nevertheless, we also observe that our voting function brings an extra benefit to the final system.