Improving Compositional Generalization in Classification Tasks via Structure Annotations

TL;DR

This paper explores how structural annotations like parse trees and entity links can improve neural models' ability to generalize compositionally in classification tasks, addressing a key challenge in AI.

Contribution

The paper introduces methods to convert sequence-to-sequence datasets into classification datasets requiring compositional generalization and demonstrates that structural hints enhance model performance.

Findings

Structural hints improve compositional generalization in Transformers.

Conversion methods enable new classification datasets for compositional tasks.

Structural annotations lead to better systematic generalization.

Abstract

Compositional generalization is the ability to generalize systematically to a new data distribution by combining known components. Although humans seem to have a great ability to generalize compositionally, state-of-the-art neural models struggle to do so. In this work, we study compositional generalization in classification tasks and present two main contributions. First, we study ways to convert a natural language sequence-to-sequence dataset to a classification dataset that also requires compositional generalization. Second, we show that providing structural hints (specifically, providing parse trees and entity links as attention masks for a Transformer model) helps compositional generalization.