Interpretable Text Embeddings and Text Similarity Explanation: A Survey

TL;DR

This survey reviews methods for creating interpretable text embeddings and explaining text similarity, highlighting current approaches, challenges, and future research opportunities in this underexplored area.

Contribution

It provides a structured overview of interpretable text embeddings and similarity explanation methods, comparing evaluation approaches and discussing open challenges.

Findings

Identifies key approaches and trade-offs in interpretability methods.

Highlights evaluation strategies for interpretability.

Outlines future research directions and open challenges.

Abstract

Text embeddings are a fundamental component in many NLP tasks, including classification, regression, clustering, and semantic search. However, despite their ubiquitous application, challenges persist in interpreting embeddings and explaining similarities between them. In this work, we provide a structured overview of methods specializing in inherently interpretable text embeddings and text similarity explanation, an underexplored research area. We characterize the main ideas, approaches, and trade-offs. We compare means of evaluation, discuss overarching lessons learned and finally identify opportunities and open challenges for future research.