Additive Large Language Models for Semi-Structured Text

TL;DR

CALM introduces an interpretable additive framework for semi-structured clinical text classification using large language models, enabling transparent predictions and meaningful visualizations while maintaining competitive performance.

Contribution

The paper presents CALM, a novel additive LLM-based framework that enhances interpretability and trust in clinical text classification without sacrificing accuracy.

Findings

CALM achieves performance comparable to traditional LLM classifiers.

CALM provides faithful explanations at patient and population levels.

CALM enables clear visualizations like component-level risk curves.

Abstract

Large Language Models have advanced clinical text classification, but their opaque predictions remain a critical barrier to practical adoption in research and clinical settings where investigators and physicians need to understand which parts of a patient's record drive risk signals. To address this challenge, we introduce \textbf{CALM}, short for \textbf{Classification with Additive Large Language Models}, an interpretable framework for semi-structured text where inputs are composed of semantically meaningful components, such as sections of an admission note or question-answer fields from an intake form. CALM predicts outcomes as the additive sum of each component's contribution, making these contributions part of the forward computation itself and enabling faithful explanations at both the patient and population level. The additive structure also enables clear visualizations, such as component-level risk curves similar to those used in generalized additive models, making the learned relationships easier to inspect and communicate. Although CALM expects semi-structured inputs, many clinical documents already have this form, and similar structure can often be automatically extracted from free-text notes. CALM achieves performance comparable to conventional LLM classifiers while improving trust, supporting quality-assurance checks, and revealing clinically meaningful patterns during model development and auditing.