Dynamic MOdularized Reasoning for Compositional Structured Explanation Generation

TL;DR

This paper introduces MORSE, a dynamic modularized neural reasoning model that enhances compositional generalization in structured explanation generation by dynamically routing inputs through specialized modules, outperforming existing methods.

Contribution

Proposes MORSE, a novel dynamic modularized reasoning framework that improves neural models' compositional generalization in structured explanation tasks.

Findings

MORSE outperforms baseline models on reasoning benchmarks.

Dynamic routing of modules enhances generalization.

Model ablations confirm the effectiveness of modular design.

Abstract

Despite the success of neural models in solving reasoning tasks, their compositional generalization capabilities remain unclear. In this work, we propose a new setting of the structured explanation generation task to facilitate compositional reasoning research. Previous works found that symbolic methods achieve superior compositionality by using pre-defined inference rules for iterative reasoning. But these approaches rely on brittle symbolic transfers and are restricted to well-defined tasks. Hence, we propose a dynamic modularized reasoning model, MORSE, to improve the compositional generalization of neural models. MORSE factorizes the inference process into a combination of modules, where each module represents a functional unit. Specifically, we adopt modularized self-attention to dynamically select and route inputs to dedicated heads, which specializes them to specific functions. We conduct experiments for increasing lengths and shapes of reasoning trees on two benchmarks to test MORSE's compositional generalization abilities, and find it outperforms competitive baselines. Model ablation and deeper analyses show the effectiveness of dynamic reasoning modules and their generalization abilities.