Zero-shot Commonsense Reasoning over Machine Imagination

TL;DR

This paper introduces Imagine, a novel zero-shot reasoning framework that enhances pre-trained language models with machine-generated visual signals, improving commonsense reasoning by mitigating textual reporting bias and boosting generalization.

Contribution

It proposes integrating machine-generated images into PLMs for zero-shot reasoning, creating a synthetic dataset, and demonstrating significant performance improvements.

Findings

Outperforms existing methods on multiple benchmarks.

Machine imagination reduces reporting bias.

Enhances generalization in zero-shot reasoning.

Abstract

Recent approaches to zero-shot commonsense reasoning have enabled Pre-trained Language Models (PLMs) to learn a broad range of commonsense knowledge without being tailored to specific situations. However, they often suffer from human reporting bias inherent in textual commonsense knowledge, leading to discrepancies in understanding between PLMs and humans. In this work, we aim to bridge this gap by introducing an additional information channel to PLMs. We propose Imagine (Machine Imagination-based Reasoning), a novel zero-shot commonsense reasoning framework designed to complement textual inputs with visual signals derived from machine-generated images. To achieve this, we enhance PLMs with imagination capabilities by incorporating an image generator into the reasoning process. To guide PLMs in effectively leveraging machine imagination, we create a synthetic pre-training dataset that simulates visual question-answering. Our extensive experiments on diverse reasoning benchmarks and analysis show that Imagine outperforms existing methods by a large margin, highlighting the strength of machine imagination in mitigating reporting bias and enhancing generalization capabilities.