CASE: Context-Aware Semantic Expansion

TL;DR

This paper introduces the task of Context-Aware Semantic Expansion (CASE), which suggests context-fitting terms for a seed word, leveraging large-scale automatic annotations and a novel neural network architecture.

Contribution

The study defines CASE, demonstrates automatic annotation harvesting from corpora, and proposes a neural network with context encoding and seed-aware attention for term suggestion.

Findings

Achieved competitive results with various context encoders.

Demonstrated effective automatic annotation harvesting.

Proposed a flexible neural network architecture for semantic expansion.

Abstract

In this paper, we define and study a new task called Context-Aware Semantic Expansion (CASE). Given a seed term in a sentential context, we aim to suggest other terms that well fit the context as the seed. CASE has many interesting applications such as query suggestion, computer-assisted writing, and word sense disambiguation, to name a few. Previous explorations, if any, only involve some similar tasks, and all require human annotations for evaluation. In this study, we demonstrate that annotations for this task can be harvested at scale from existing corpora, in a fully automatic manner. On a dataset of 1.8 million sentences thus derived, we propose a network architecture that encodes the context and seed term separately before suggesting alternative terms. The context encoder in this architecture can be easily extended by incorporating seed-aware attention. Our experiments demonstrate that competitive results are achieved with appropriate choices of context encoder and attention scoring function.