Iterative Recursive Attention Model for Interpretable Sequence Classification

TL;DR

This paper introduces an iterative recursive attention model that enhances interpretability in sequence classification tasks by building incremental input representations through recursive attention, achieving competitive performance.

Contribution

The paper presents a novel recursive attention mechanism that improves interpretability in sequence classification without sacrificing accuracy.

Findings

Model achieves near state-of-the-art performance.

Effectively identifies and combines input aspects.

Provides interpretable inference steps.

Abstract

Natural language processing has greatly benefited from the introduction of the attention mechanism. However, standard attention models are of limited interpretability for tasks that involve a series of inference steps. We describe an iterative recursive attention model, which constructs incremental representations of input data through reusing results of previously computed queries. We train our model on sentiment classification datasets and demonstrate its capacity to identify and combine different aspects of the input in an easily interpretable manner, while obtaining performance close to the state of the art.