Do Language Models Perform Generalizable Commonsense Inference?

TL;DR

This paper investigates how well pretrained language models can perform generalizable commonsense inference across different knowledge graphs, focusing on their knowledge capacity, transferability, and induction abilities.

Contribution

It provides a comprehensive analysis of LMs' ability to generalize commonsense knowledge, highlighting their strengths and limitations in transferability and induction.

Findings

LMS adapt to different CKG schemas but struggle with new relations.

They generalize well to unseen subjects but less to novel objects.

Future work needed to enhance transferability and induction.

Abstract

Inspired by evidence that pretrained language models (LMs) encode commonsense knowledge, recent work has applied LMs to automatically populate commonsense knowledge graphs (CKGs). However, there is a lack of understanding on their generalization to multiple CKGs, unseen relations, and novel entities. This paper analyzes the ability of LMs to perform generalizable commonsense inference, in terms of knowledge capacity, transferability, and induction. Our experiments with these three aspects show that: (1) LMs can adapt to different schemas defined by multiple CKGs but fail to reuse the knowledge to generalize to new relations. (2) Adapted LMs generalize well to unseen subjects, but less so on novel objects. Future work should investigate how to improve the transferability and induction of commonsense mining from LMs.