Creating Causal Embeddings for Question Answering with Minimal Supervision

TL;DR

This paper introduces a method for creating task-specific causal embeddings from minimal supervision, improving question answering performance by explicitly modeling causality.

Contribution

It presents a novel approach to generate causal embeddings using bootstrapped cause-effect pairs and incorporates them into QA reranking, with significant performance gains.

Findings

Causal embeddings improve QA accuracy.

Explicit causality modeling outperforms lexical similarity.

Method achieves 37.3% P@1 in causal QA.

Abstract

A common model for question answering (QA) is that a good answer is one that is closely related to the question, where relatedness is often determined using general-purpose lexical models such as word embeddings. We argue that a better approach is to look for answers that are related to the question in a relevant way, according to the information need of the question, which may be determined through task-specific embeddings. With causality as a use case, we implement this insight in three steps. First, we generate causal embeddings cost-effectively by bootstrapping cause-effect pairs extracted from free text using a small set of seed patterns. Second, we train dedicated embeddings over this data, by using task-specific contexts, i.e., the context of a cause is its effect. Finally, we extend a state-of-the-art reranking approach for QA to incorporate these causal embeddings. We evaluate the causal embedding models both directly with a casual implication task, and indirectly, in a downstream causal QA task using data from Yahoo! Answers. We show that explicitly modeling causality improves performance in both tasks. In the QA task our best model achieves 37.3% P@1, significantly outperforming a strong baseline by 7.7% (relative).