TabSHAP

TL;DR

TabSHAP is a novel interpretability framework for LLM-based tabular classifiers that attributes local decision logic by quantifying feature distribution impacts, improving faithfulness over existing methods.

Contribution

It introduces a Shapley-style estimator using Jensen-Shannon divergence for faithful local interpretability of LLM classifiers on tabular data.

Findings

TabSHAP outperforms random baselines and XGBoost proxies in faithfulness.

It effectively isolates critical diagnostic features in benchmark datasets.

Ablation studies show the impact of different divergence metrics on attribution quality.

Abstract

Large Language Models (LLMs) fine-tuned on serialized tabular data are emerging as powerful alternatives to traditional tree-based models, particularly for heterogeneous or context-rich datasets. However, their deployment in high-stakes domains is hindered by a lack of faithful interpretability; existing methods often rely on global linear proxies or scalar probability shifts that fail to capture the model's full probabilistic uncertainty. In this work, we introduce TabSHAP, a model-agnostic interpretability framework designed to directly attribute local query decision logic in LLM-based tabular classifiers. By adapting a Shapley-style sampled-coalition estimator with Jensen-Shannon divergence between full-input and masked-input class distributions, TabSHAP quantifies the distributional impact of each feature rather than simple prediction flips. To align with tabular semantics, we mask at the level of serialized key:value fields (atomic in the prompt string), not individual subword tokens. Experimental validation on the Adult Income and Heart Disease benchmarks demonstrates that TabSHAP isolates critical diagnostic features, achieving significantly higher faithfulness than random baselines and XGBoost proxies. We further run a distance-metric ablation on the same test instances and TabSHAP settings: attributions are recomputed with KL or L1 replacing JSD in the similarity step (results cached per metric), and we compare deletion faithfulness across all three.