Improving Numerical Reasoning Skills in the Modular Approach for Complex Question Answering on Text

TL;DR

This paper enhances Neural Module Networks for complex text question answering by making them question-aware and better capturing entity-number relationships, leading to improved numerical reasoning performance.

Contribution

It introduces techniques to improve NMNs' numerical reasoning by incorporating question-awareness and entity-number relationship modeling.

Findings

Outperforms original NMNs by 3.0 points in F1 score on DROP dataset

Improves numerical reasoning accuracy in complex question answering

Enhances the interpretability of NMNs in reasoning tasks

Abstract

Numerical reasoning skills are essential for complex question answering (CQA) over text. It requires opertaions including counting, comparison, addition and subtraction. A successful approach to CQA on text, Neural Module Networks (NMNs), follows the programmer-interpreter paradigm and leverages specialised modules to perform compositional reasoning. However, the NMNs framework does not consider the relationship between numbers and entities in both questions and paragraphs. We propose effective techniques to improve NMNs' numerical reasoning capabilities by making the interpreter question-aware and capturing the relationship between entities and numbers. On the same subset of the DROP dataset for CQA on text, experimental results show that our additions outperform the original NMNs by 3.0 points for the overall F1 score.