Differentiable Open-Ended Commonsense Reasoning

TL;DR

This paper introduces DrFact, a differentiable model for open-ended commonsense reasoning that answers questions without predefined choices, demonstrating significant improvements over baselines on adapted benchmarks.

Contribution

The paper presents DrFact, a novel differentiable model designed for multi-hop reasoning over knowledge facts in open-ended commonsense questions, advancing realistic reasoning applications.

Findings

DrFact outperforms baseline methods significantly.

OpenCSR is feasible with the proposed approach.

New crowd-sourced answers enhance evaluation.

Abstract

Current commonsense reasoning research focuses on developing models that use commonsense knowledge to answer multiple-choice questions. However, systems designed to answer multiple-choice questions may not be useful in applications that do not provide a small list of candidate answers to choose from. As a step towards making commonsense reasoning research more realistic, we propose to study open-ended commonsense reasoning (OpenCSR) -- the task of answering a commonsense question without any pre-defined choices -- using as a resource only a corpus of commonsense facts written in natural language. OpenCSR is challenging due to a large decision space, and because many questions require implicit multi-hop reasoning. As an approach to OpenCSR, we propose DrFact, an efficient Differentiable model for multi-hop Reasoning over knowledge Facts. To evaluate OpenCSR methods, we adapt several popular commonsense reasoning benchmarks, and collect multiple new answers for each test question via crowd-sourcing. Experiments show that DrFact outperforms strong baseline methods by a large margin.