Zero-Shot Classification by Logical Reasoning on Natural Language Explanations

TL;DR

This paper introduces CLORE, a framework that performs zero-shot classification by parsing and reasoning on natural language explanations, improving accuracy especially on tasks requiring logical reasoning and reducing linguistic biases.

Contribution

CLORE is the first method to explicitly parse explanations into logical structures for reasoning, extending zero-shot classification to both language and visual data.

Findings

CLORE outperforms baselines on explanation-based zero-shot benchmarks.

It achieves higher scores on tasks requiring complex logical reasoning.

CLORE is less affected by linguistic biases than previous methods.

Abstract

Humans can classify data of an unseen category by reasoning on its language explanations. This ability is owing to the compositional nature of language: we can combine previously seen attributes to describe the new category. For example, we might describe a sage thrasher as "it has a slim straight relatively short bill, yellow eyes and a long tail", so that others can use their knowledge of attributes "slim straight relatively short bill", "yellow eyes" and "long tail" to recognize a sage thrasher. Inspired by this observation, in this work we tackle zero-shot classification task by logically parsing and reasoning on natural language expla-nations. To this end, we propose the framework CLORE (Classification by LOgical Reasoning on Explanations). While previous methods usually regard textual information as implicit features, CLORE parses explanations into logical structures and then explicitly reasons along thess structures on the input to produce a classification score. Experimental results on explanation-based zero-shot classification benchmarks demonstrate that CLORE is superior to baselines, which we further show mainly comes from higher scores on tasks requiring more logical reasoning. We also demonstrate that our framework can be extended to zero-shot classification on visual modality. Alongside classification decisions, CLORE can provide the logical parsing and reasoning process as a clear form of rationale. Through empirical analysis we demonstrate that CLORE is also less affected by linguistic biases than baselines.