Explanation based Bias Decoupling Regularization for Natural Language Inference

TL;DR

This paper introduces EBD-Reg, a novel regularization method that uses human explanations to help Transformer models identify task-relevant features and reduce reliance on dataset biases, improving out-of-distribution inference in natural language inference.

Contribution

It presents a new interpretable regularization technique that leverages human explanations to explicitly decouple biases from relevant features in Transformer-based NLI models.

Findings

EBD-Reg effectively guides models to focus on task-relevant keywords.

The method significantly improves out-of-distribution inference performance.

EBD-Reg outperforms existing debiasing approaches in experiments.

Abstract

The robustness of Transformer-based Natural Language Inference encoders is frequently compromised as they tend to rely more on dataset biases than on the intended task-relevant features. Recent studies have attempted to mitigate this by reducing the weight of biased samples during the training process. However, these debiasing methods primarily focus on identifying which samples are biased without explicitly determining the biased components within each case. This limitation restricts those methods' capability in out-of-distribution inference. To address this issue, we aim to train models to adopt the logic humans use in explaining causality. We propose a simple, comprehensive, and interpretable method: Explanation based Bias Decoupling Regularization (EBD-Reg). EBD-Reg employs human explanations as criteria, guiding the encoder to establish a tripartite parallel supervision of Distinguishing, Decoupling and Aligning. This method enables encoders to identify and focus on keywords that represent the task-relevant features during inference, while discarding the residual elements acting as biases. Empirical evidence underscores that EBD-Reg effectively guides various Transformer-based encoders to decouple biases through a human-centric lens, significantly surpassing other methods in terms of out-of-distribution inference capabilities.