Unsupervised Deep Structured Semantic Models for Commonsense Reasoning

TL;DR

This paper introduces unsupervised neural models based on DSSM to improve commonsense reasoning in NLP, effectively capturing context and co-reference for tasks like Winograd Schema and Pronoun Disambiguation.

Contribution

It presents two novel unsupervised neural network models leveraging DSSM for commonsense reasoning, outperforming previous state-of-the-art methods.

Findings

Models effectively capture contextual and co-reference information.

Significant improvement over previous approaches on WSC and PDP tasks.

Demonstrates potential of unsupervised learning for commonsense reasoning.

Abstract

Commonsense reasoning is fundamental to natural language understanding. While traditional methods rely heavily on human-crafted features and knowledge bases, we explore learning commonsense knowledge from a large amount of raw text via unsupervised learning. We propose two neural network models based on the Deep Structured Semantic Models (DSSM) framework to tackle two classic commonsense reasoning tasks, Winograd Schema challenges (WSC) and Pronoun Disambiguation (PDP). Evaluation shows that the proposed models effectively capture contextual information in the sentence and co-reference information between pronouns and nouns, and achieve significant improvement over previous state-of-the-art approaches.