TT-XAI: Trustworthy Clinical Text Explanations via Keyword Distillation and LLM Reasoning

TL;DR

TT-XAI is a framework that enhances clinical text classification and interpretability by distilling key information and guiding large language models to produce trustworthy, concise explanations for electronic health records.

Contribution

It introduces a novel keyword distillation method combined with LLM reasoning to improve trustworthiness and interpretability of clinical language models.

Findings

Keyword distillation improves classifier performance and explanation fidelity.

Guided LLM reasoning produces more clinically relevant explanations.

Evaluation shows consistent preference for keyword-augmented explanations.

Abstract

Clinical language models often struggle to provide trustworthy predictions and explanations when applied to lengthy, unstructured electronic health records (EHRs). This work introduces TT-XAI, a lightweight and effective framework that improves both classification performance and interpretability through domain-aware keyword distillation and reasoning with large language models (LLMs). First, we demonstrate that distilling raw discharge notes into concise keyword representations significantly enhances BERT classifier performance and improves local explanation fidelity via a focused variant of LIME. Second, we generate chain-of-thought clinical explanations using keyword-guided prompts to steer LLMs, producing more concise and clinically relevant reasoning. We evaluate explanation quality using deletion-based fidelity metrics, self-assessment via LLaMA-3 scoring, and a blinded human study with domain experts. All evaluation modalities consistently favor the keyword-augmented method, confirming that distillation enhances both machine and human interpretability. TT-XAI offers a scalable pathway toward trustworthy, auditable AI in clinical decision support.