TopEx: Topic-based Explanations for Model Comparison

TL;DR

TopEx introduces a model-agnostic explanation method using topics to facilitate meaningful comparison of language models, addressing the limitations of current explanation techniques.

Contribution

The paper presents TopEx, a novel explanation approach that enables fair comparison of language models through the use of model-agnostic topics.

Findings

TopEx effectively identifies similarities between DistilRoBERTa and GPT-2.

TopEx reveals differences in model behavior across NLP tasks.

The method simplifies explanation complexity for human understanding.

Abstract

Meaningfully comparing language models is challenging with current explanation methods. Current explanations are overwhelming for humans due to large vocabularies or incomparable across models. We present TopEx, an explanation method that enables a level playing field for comparing language models via model-agnostic topics. We demonstrate how TopEx can identify similarities and differences between DistilRoBERTa and GPT-2 on a variety of NLP tasks.