A General Framework for Producing Interpretable Semantic Text Embeddings

TL;DR

This paper introduces CQG-MBQA, a versatile framework that generates interpretable semantic text embeddings using discriminative yes/no questions, achieving high quality and interpretability across diverse NLP tasks.

Contribution

The paper presents a novel, general framework combining contrastive question generation and multi-task binary question answering to produce interpretable embeddings without relying on expert-crafted questions.

Findings

Achieves embedding quality comparable to black-box models.

Outperforms existing interpretable embedding methods.

Demonstrates effectiveness across multiple NLP tasks.

Abstract

Semantic text embedding is essential to many tasks in Natural Language Processing (NLP). While black-box models are capable of generating high-quality embeddings, their lack of interpretability limits their use in tasks that demand transparency. Recent approaches have improved interpretability by leveraging domain-expert-crafted or LLM-generated questions, but these methods rely heavily on expert input or well-prompt design, which restricts their generalizability and ability to generate discriminative questions across a wide range of tasks. To address these challenges, we introduce \algo{CQG-MBQA} (Contrastive Question Generation - Multi-task Binary Question Answering), a general framework for producing interpretable semantic text embeddings across diverse tasks. Our framework systematically generates highly discriminative, low cognitive load yes/no questions through the \algo{CQG} method and answers them efficiently with the \algo{MBQA} model, resulting in interpretable embeddings in a cost-effective manner. We validate the effectiveness and interpretability of \algo{CQG-MBQA} through extensive experiments and ablation studies, demonstrating that it delivers embedding quality comparable to many advanced black-box models while maintaining inherently interpretability. Additionally, \algo{CQG-MBQA} outperforms other interpretable text embedding methods across various downstream tasks.