Learning when to skim and when to read

TL;DR

This paper introduces methods to optimize computational efficiency in NLP models by selectively choosing when to use a fast or slow classifier, demonstrating significant efficiency improvements.

Contribution

It proposes two novel approaches for reducing unnecessary computation by leveraging a baseline classifier and a secondary decision network.

Findings

Significant efficiency gains in sentiment classification tasks.

Effective probability-threshold method for computational reduction.

Secondary decision network further improves efficiency.

Abstract

Many recent advances in deep learning for natural language processing have come at increasing computational cost, but the power of these state-of-the-art models is not needed for every example in a dataset. We demonstrate two approaches to reducing unnecessary computation in cases where a fast but weak baseline classier and a stronger, slower model are both available. Applying an AUC-based metric to the task of sentiment classification, we find significant efficiency gains with both a probability-threshold method for reducing computational cost and one that uses a secondary decision network.