Mind Reasoning Manners: Enhancing Type Perception for Generalized Zero-shot Logical Reasoning over Text

TL;DR

This paper introduces a new benchmark ZsLR for generalized zero-shot logical reasoning and proposes the type-aware model TaCo, which enhances reasoning type perception and outperforms existing methods in zero-shot and full-data settings.

Contribution

The paper presents a novel zero-shot reasoning benchmark ZsLR and a type-aware model TaCo that improves reasoning type perception and generalization.

Findings

TaCo outperforms state-of-the-art methods in zero-shot reasoning.

TaCo demonstrates strong generalization on other logical reasoning datasets.

ZsLR effectively evaluates zero-shot logical reasoning capabilities.

Abstract

Logical reasoning task involves diverse types of complex reasoning over text, based on the form of multiple-choice question answering. Given the context, question and a set of options as the input, previous methods achieve superior performances on the full-data setting. However, the current benchmark dataset has the ideal assumption that the reasoning type distribution on the train split is close to the test split, which is inconsistent with many real application scenarios. To address it, there remain two problems to be studied: (1) How is the zero-shot capability of the models (train on seen types and test on unseen types)? (2) How to enhance the perception of reasoning types for the models? For problem 1, we propose a new benchmark for generalized zero-shot logical reasoning, named ZsLR. It includes six splits based on the three type sampling strategies. For problem 2, a type-aware model TaCo is proposed. It utilizes both the heuristic input reconstruction and the contrastive learning to improve the type perception in the global representation. Extensive experiments on both the zero-shot and full-data settings prove the superiority of TaCo over the state-of-the-art methods. Also, we experiment and verify the generalization capability of TaCo on other logical reasoning dataset.