Interlocking-free Selective Rationalization Through Genetic-based Learning

TL;DR

This paper introduces GenSPP, a novel architecture for selective rationalization that eliminates interlocking issues by using genetic search for disjoint training, leading to improved performance without additional learning overhead.

Contribution

GenSPP is the first interlocking-free selective rationalization model that employs genetic search for disjoint training, avoiding common pitfalls of existing cooperative systems.

Findings

Outperforms state-of-the-art competitors on benchmarks

Eliminates interlocking without additional training overhead

Effective on synthetic and real-world data

Abstract

A popular end-to-end architecture for selective rationalization is the select-then-predict pipeline, comprising a generator to extract highlights fed to a predictor. Such a cooperative system suffers from suboptimal equilibrium minima due to the dominance of one of the two modules, a phenomenon known as interlocking. While several contributions aimed at addressing interlocking, they only mitigate its effect, often by introducing feature-based heuristics, sampling, and ad-hoc regularizations. We present GenSPP, the first interlocking-free architecture for selective rationalization that does not require any learning overhead, as the above-mentioned. GenSPP avoids interlocking by performing disjoint training of the generator and predictor via genetic global search. Experiments on a synthetic and a real-world benchmark show that our model outperforms several state-of-the-art competitors.