Text Classification with Few Examples using Controlled Generalization

TL;DR

This paper introduces a method for text classification with limited training data, using sparse pre-trained representations and a combination of neural networks to improve generalization in low-data scenarios.

Contribution

It proposes a novel approach that leverages sparse pre-trained features and a specialized neural network architecture for better low-data text classification.

Findings

Outperforms existing methods in low-data settings

Maintains competitive accuracy with full datasets

Effective use of task-specific semantic vectors

Abstract

Training data for text classification is often limited in practice, especially for applications with many output classes or involving many related classification problems. This means classifiers must generalize from limited evidence, but the manner and extent of generalization is task dependent. Current practice primarily relies on pre-trained word embeddings to map words unseen in training to similar seen ones. Unfortunately, this squishes many components of meaning into highly restricted capacity. Our alternative begins with sparse pre-trained representations derived from unlabeled parsed corpora; based on the available training data, we select features that offers the relevant generalizations. This produces task-specific semantic vectors; here, we show that a feed-forward network over these vectors is especially effective in low-data scenarios, compared to existing state-of-the-art methods. By further pairing this network with a convolutional neural network, we keep this edge in low data scenarios and remain competitive when using full training sets.