Discriminative Information Retrieval for Knowledge Discovery

TL;DR

This paper introduces a discriminative IR framework using linguistic features to enhance recall in knowledge discovery tasks like answer passage retrieval, significantly improving initial QA candidate selection.

Contribution

It formalizes a linear feature-based IR approach for knowledge discovery, demonstrating substantial recall improvements in question answering tasks.

Findings

44% improvement in recall for candidate triage in QA

Framework effectively captures various knowledge discovery tasks

Utilizes linguistic features for discriminative IR

Abstract

We propose a framework for discriminative Information Retrieval (IR) atop linguistic features, trained to improve the recall of tasks such as answer candidate passage retrieval, the initial step in text-based Question Answering (QA). We formalize this as an instance of linear feature-based IR (Metzler and Croft, 2007), illustrating how a variety of knowledge discovery tasks are captured under this approach, leading to a 44% improvement in recall for candidate triage for QA.