OPERA:Operation-Pivoted Discrete Reasoning over Text

TL;DR

OPERA introduces a novel framework combining symbolic operations with neural modules to enhance discrete reasoning and interpretability in machine reading comprehension tasks, demonstrating strong performance and explainability on benchmark datasets.

Contribution

It proposes OPERA, a new operation-pivoted framework that integrates symbolic operations with neural modules for improved reasoning and interpretability in MRC.

Findings

OPERA achieves competitive results on DROP and RACENum datasets.

The framework demonstrates enhanced interpretability of reasoning processes.

Extensive analysis confirms the reasoning ability of OPERA.

Abstract

Machine reading comprehension (MRC) that requires discrete reasoning involving symbolic operations, e.g., addition, sorting, and counting, is a challenging task. According to this nature, semantic parsing-based methods predict interpretable but complex logical forms. However, logical form generation is nontrivial and even a little perturbation in a logical form will lead to wrong answers. To alleviate this issue, multi-predictor -based methods are proposed to directly predict different types of answers and achieve improvements. However, they ignore the utilization of symbolic operations and encounter a lack of reasoning ability and interpretability. To inherit the advantages of these two types of methods, we propose OPERA, an operation-pivoted discrete reasoning framework, where lightweight symbolic operations (compared with logical forms) as neural modules are utilized to facilitate the reasoning ability and interpretability. Specifically, operations are first selected and then softly executed to simulate the answer reasoning procedure. Extensive experiments on both DROP and RACENum datasets show the reasoning ability of OPERA. Moreover, further analysis verifies its interpretability.