Commonsense mining as knowledge base completion? A study on the impact of novelty

TL;DR

This paper investigates whether knowledge base completion models can effectively mine novel commonsense knowledge from raw text, emphasizing the importance of novelty in predicted triples and analyzing the challenges involved.

Contribution

It introduces the concept of novelty in predicted triples for commonsense knowledge mining and demonstrates that a simple baseline can outperform previous methods in predicting more novel knowledge.

Findings

Simple baseline outperforms previous state-of-the-art in predicting novel triples.

Novelty of triples is a crucial factor in evaluating knowledge base completion.

Mining highly novel commonsense knowledge remains a challenging task.

Abstract

Commonsense knowledge bases such as ConceptNet represent knowledge in the form of relational triples. Inspired by the recent work by Li et al., we analyse if knowledge base completion models can be used to mine commonsense knowledge from raw text. We propose novelty of predicted triples with respect to the training set as an important factor in interpreting results. We critically analyse the difficulty of mining novel commonsense knowledge, and show that a simple baseline method outperforms the previous state of the art on predicting more novel.