The Effect of Model Size on LLM Post-hoc Explainability via LIME

TL;DR

This study investigates how increasing the size of large language models affects the quality of LIME explanations, revealing that larger models do not necessarily produce more plausible explanations despite better performance.

Contribution

It provides the first systematic analysis of the relationship between model size and LIME explanation quality across different NLP tasks.

Findings

Larger models do not yield more plausible explanations.

Model size correlates with improved performance but not explanation plausibility.

Faithfulness metrics may have limitations in NLI contexts.

Abstract

Large language models (LLMs) are becoming bigger to boost performance. However, little is known about how explainability is affected by this trend. This work explores LIME explanations for DeBERTaV3 models of four different sizes on natural language inference (NLI) and zero-shot classification (ZSC) tasks. We evaluate the explanations based on their faithfulness to the models' internal decision processes and their plausibility, i.e. their agreement with human explanations. The key finding is that increased model size does not correlate with plausibility despite improved model performance, suggesting a misalignment between the LIME explanations and the models' internal processes as model size increases. Our results further suggest limitations regarding faithfulness metrics in NLI contexts.