A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations

TL;DR

This paper introduces a deep neural architecture that matches sentences by leveraging multiple positional representations generated by Bi-LSTM, effectively capturing contextualized local information for improved performance in tasks like question answering.

Contribution

The paper proposes a novel deep model that uses multiple positional sentence representations with Bi-LSTM for better sentence matching, surpassing existing single or multi-granularity approaches.

Findings

Outperforms existing models on question answering tasks

Effectively captures contextualized local information

Demonstrates superiority in sentence completion

Abstract

Matching natural language sentences is central for many applications such as information retrieval and question answering. Existing deep models rely on a single sentence representation or multiple granularity representations for matching. However, such methods cannot well capture the contextualized local information in the matching process. To tackle this problem, we present a new deep architecture to match two sentences with multiple positional sentence representations. Specifically, each positional sentence representation is a sentence representation at this position, generated by a bidirectional long short term memory (Bi-LSTM). The matching score is finally produced by aggregating interactions between these different positional sentence representations, through $k$-Max pooling and a multi-layer perceptron. Our model has several advantages: (1) By using Bi-LSTM, rich context of the whole sentence is leveraged to capture the contextualized local information in each positional sentence representation; (2) By matching with multiple positional sentence representations, it is flexible to aggregate different important contextualized local information in a sentence to support the matching; (3) Experiments on different tasks such as question answering and sentence completion demonstrate the superiority of our model.