Sensivity of LLMs' Explanations to the Training Randomness:Context, Class & Task Dependencies

TL;DR

This paper investigates how the explanations of transformer-based language models vary with training randomness, focusing on the influence of context, class, and task, revealing significant dependencies.

Contribution

It provides a systematic analysis of the factors affecting explanation stability in LLMs, highlighting the varying impact of context, class, and task on explanation sensitivity.

Findings

Explanations are least sensitive to context variations.

Class differences moderately affect explanation stability.

Task complexity significantly influences explanation variability.

Abstract

Transformer models are now a cornerstone in natural language processing. Yet, explaining their decisions remains a challenge. It was shown recently that the same model trained on the same data with a different randomness can lead to very different explanations. In this paper, we investigate how the (syntactic) context, the classes to be learned and the tasks influence this explanations' sensitivity to randomness. We show that they all have statistically significant impact: smallest for the (syntactic) context, medium for the classes and largest for the tasks.