Modularized Transfer Learning with Multiple Knowledge Graphs for Zero-shot Commonsense Reasoning

TL;DR

This paper introduces a modular zero-shot transfer learning framework that leverages multiple knowledge graphs to enhance commonsense reasoning across diverse benchmarks, reducing interference among knowledge sources.

Contribution

It proposes a novel modular knowledge aggregation method for multi-source zero-shot transfer learning in commonsense reasoning, addressing knowledge interference issues.

Findings

Improves performance on five commonsense reasoning benchmarks.

Effectively utilizes multiple knowledge graphs synergistically.

Reduces knowledge interference among different sources.

Abstract

Commonsense reasoning systems should be able to generalize to diverse reasoning cases. However, most state-of-the-art approaches depend on expensive data annotations and overfit to a specific benchmark without learning how to perform general semantic reasoning. To overcome these drawbacks, zero-shot QA systems have shown promise as a robust learning scheme by transforming a commonsense knowledge graph (KG) into synthetic QA-form samples for model training. Considering the increasing type of different commonsense KGs, this paper aims to extend the zero-shot transfer learning scenario into multiple-source settings, where different KGs can be utilized synergetically. Towards this goal, we propose to mitigate the loss of knowledge from the interference among the different knowledge sources, by developing a modular variant of the knowledge aggregation as a new zero-shot commonsense reasoning framework. Results on five commonsense reasoning benchmarks demonstrate the efficacy of our framework, improving the performance with multiple KGs.