An Information-Theoretic Framework for Comparing Voice and Text Explainability

TL;DR

This paper presents an information-theoretic framework to compare voice and text explanations in AI, analyzing their impact on user understanding and trust, and providing a basis for designing multimodal explainability systems.

Contribution

It introduces a novel communication channel model for explanation modalities and develops a simulation framework to evaluate their effectiveness and trade-offs.

Findings

Text explanations have higher comprehension efficiency.

Voice explanations improve trust calibration.

Analogy-based explanations offer the best overall trade-off.

Abstract

Explainable Artificial Intelligence (XAI) aims to make machine learning models transparent and trustworthy, yet most current approaches communicate explanations visually or through text. This paper introduces an information theoretic framework for analyzing how explanation modality specifically, voice versus text affects user comprehension and trust calibration in AI systems. The proposed model treats explanation delivery as a communication channel between model and user, characterized by metrics for information retention, comprehension efficiency (CE), and trust calibration error (T CE). A simulation framework implemented in Python was developed to evaluate these metrics using synthetic SHAP based feature attributions across multiple modality style configurations (brief, detailed, and analogy based). Results demonstrate that text explanations achieve higher comprehension efficiency, while voice explanations yield improved trust calibration, with analogy based delivery achieving the best overall trade off. This framework provides a reproducible foundation for designing and benchmarking multimodal explainability systems and can be extended to empirical studies using real SHAP or LIME outputs on open datasets such as the UCI Credit Approval or Kaggle Financial Transactions datasets.