Subgraph Retrieval Enhanced by Graph-Text Alignment for Commonsense Question Answering

TL;DR

SEPTA is a novel framework for commonsense question answering that improves subgraph retrieval and graph-text alignment, leading to better reasoning by addressing limitations of rule-based subgraph extraction and modality misalignment.

Contribution

The paper introduces SEPTA, a framework that uses a BFS-style sampling and bidirectional contrastive learning to enhance subgraph retrieval and graph-text alignment in commonsense QA.

Findings

Outperforms previous methods on five datasets

Improves subgraph retrieval accuracy

Enhances knowledge fusion effectiveness

Abstract

Commonsense question answering is a crucial task that requires machines to employ reasoning according to commonsense. Previous studies predominantly employ an extracting-and-modeling paradigm to harness the information in KG, which first extracts relevant subgraphs based on pre-defined rules and then proceeds to design various strategies aiming to improve the representations and fusion of the extracted structural knowledge. Despite their effectiveness, there are still two challenges. On one hand, subgraphs extracted by rule-based methods may have the potential to overlook critical nodes and result in uncontrollable subgraph size. On the other hand, the misalignment between graph and text modalities undermines the effectiveness of knowledge fusion, ultimately impacting the task performance. To deal with the problems above, we propose a novel framework: \textbf{S}ubgraph R\textbf{E}trieval Enhanced by Gra\textbf{P}h-\textbf{T}ext \textbf{A}lignment, named \textbf{SEPTA}. Firstly, we transform the knowledge graph into a database of subgraph vectors and propose a BFS-style subgraph sampling strategy to avoid information loss, leveraging the analogy between BFS and the message-passing mechanism. In addition, we propose a bidirectional contrastive learning approach for graph-text alignment, which effectively enhances both subgraph retrieval and knowledge fusion. Finally, all the retrieved information is combined for reasoning in the prediction module. Extensive experiments on five datasets demonstrate the effectiveness and robustness of our framework.