Common Sense Knowledge Learning for Open Vocabulary Neural Reasoning: A First View into Chronic Disease Literature

TL;DR

This paper explores training neural language models on common sense knowledge bases to improve reasoning over scientific literature related to chronic diseases, demonstrating promising results and insights into learned semantic regularities.

Contribution

It introduces a novel approach of using common sense KBs to enhance open vocabulary reasoning in scientific literature, specifically targeting chronic disease knowledge.

Findings

Neural language models performed significantly in knowledge inference tasks.

Models learned semantic regularities relevant to scientific reasoning.

Potential benefits for non-communicable disease research identified.

Abstract

In this paper, we address reasoning tasks from open vocabulary Knowledge Bases (openKBs) using state-of-the-art Neural Language Models (NLMs) with applications in scientific literature. For this purpose, self-attention based NLMs are trained using a common sense KB as a source task. The NLMs are then tested on a target KB for open vocabulary reasoning tasks involving scientific knowledge related to the most prevalent chronic diseases (also known as non-communicable diseases, NCDs). Our results identified NLMs that performed consistently and with significance in knowledge inference for both source and target tasks. Furthermore, in our analysis by inspection we discussed the semantic regularities and reasoning capabilities learned by the models, while showing a first insight into the potential benefits of our approach to aid NCD research.