On the Impact of Temporal Concept Drift on Model Explanations
Zhixue Zhao, George Chrysostomou, Kalina Bontcheva, Nikolaos Aletras

TL;DR
This paper investigates how temporal concept drift affects the faithfulness of model explanations in NLP, revealing variability across methods and the robustness of select-then-predict models in asynchronous settings.
Contribution
It provides a comprehensive analysis of explanation faithfulness under temporal variation, highlighting the inconsistent behavior of attribution methods and robustness of select-then-predict models.
Findings
Faithfulness varies with temporal drift and attribution method.
Attention-based explanations are most robust across datasets.
Select-then-predict models show small performance degradation in asynchronous settings.
Abstract
Explanation faithfulness of model predictions in natural language processing is typically evaluated on held-out data from the same temporal distribution as the training data (i.e. synchronous settings). While model performance often deteriorates due to temporal variation (i.e. temporal concept drift), it is currently unknown how explanation faithfulness is impacted when the time span of the target data is different from the data used to train the model (i.e. asynchronous settings). For this purpose, we examine the impact of temporal variation on model explanations extracted by eight feature attribution methods and three select-then-predict models across six text classification tasks. Our experiments show that (i)faithfulness is not consistent under temporal variations across feature attribution methods (e.g. it decreases or increases depending on the method), with an attention-based…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsData Stream Mining Techniques · Topic Modeling · Stock Market Forecasting Methods
