End-to-end Learning for Short Text Expansion

TL;DR

This paper introduces an end-to-end deep memory network approach for short text expansion, automatically improving short text understanding by leveraging relevant information from longer documents, and demonstrates superior performance over traditional methods.

Contribution

The paper presents a novel deep memory network that learns to expand short texts automatically, optimizing for specific learning tasks without relying on heuristics.

Findings

Deep memory network outperforms classical expansion methods

Significant improvements in short text classification accuracy

Effective automatic relevance learning from long documents

Abstract

Effectively making sense of short texts is a critical task for many real world applications such as search engines, social media services, and recommender systems. The task is particularly challenging as a short text contains very sparse information, often too sparse for a machine learning algorithm to pick up useful signals. A common practice for analyzing short text is to first expand it with external information, which is usually harvested from a large collection of longer texts. In literature, short text expansion has been done with all kinds of heuristics. We propose an end-to-end solution that automatically learns how to expand short text to optimize a given learning task. A novel deep memory network is proposed to automatically find relevant information from a collection of longer documents and reformulate the short text through a gating mechanism. Using short text classification as a demonstrating task, we show that the deep memory network significantly outperforms classical text expansion methods with comprehensive experiments on real world data sets.