Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts

TL;DR

This paper introduces MoKGE, a mixture of knowledge graph experts approach that enhances diversity in generative commonsense reasoning without sacrificing accuracy, by leveraging multiple reasoning paths on knowledge graphs.

Contribution

The paper presents a novel MoE-based method for diversifying generative commonsense reasoning using knowledge graphs, improving output variety while maintaining accuracy.

Findings

MoKGE significantly increases diversity in generated outputs.

MoKGE maintains comparable accuracy to existing methods.

Human evaluations confirm improved diversity.

Abstract

Generative commonsense reasoning (GCR) in natural language is to reason about the commonsense while generating coherent text. Recent years have seen a surge of interest in improving the generation quality of commonsense reasoning tasks. Nevertheless, these approaches have seldom investigated diversity in the GCR tasks, which aims to generate alternative explanations for a real-world situation or predict all possible outcomes. Diversifying GCR is challenging as it expects to generate multiple outputs that are not only semantically different but also grounded in commonsense knowledge. In this paper, we propose MoKGE, a novel method that diversifies the generative reasoning by a mixture of expert (MoE) strategy on commonsense knowledge graphs (KG). A set of knowledge experts seek diverse reasoning on KG to encourage various generation outputs. Empirical experiments demonstrated that MoKGE can significantly improve the diversity while achieving on par performance on accuracy on two GCR benchmarks, based on both automatic and human evaluations.