Incremental Sequence Classification with Temporal Consistency

TL;DR

This paper introduces a novel loss function for incremental sequence classification that enforces temporal consistency, leading to improved data efficiency and accuracy in text classification and language model verification tasks.

Contribution

It proposes a new training method based on temporal consistency inspired by reinforcement learning, enhancing incremental sequence classification performance.

Findings

Improved predictive accuracy on benchmark text classification datasets.

Enhanced ability to verify language model generations with fewer tokens.

Better data efficiency in incremental learning scenarios.

Abstract

We address the problem of incremental sequence classification, where predictions are updated as new elements in the sequence are revealed. Drawing on temporal-difference learning from reinforcement learning, we identify a temporal-consistency condition that successive predictions should satisfy. We leverage this condition to develop a novel loss function for training incremental sequence classifiers. Through a concrete example, we demonstrate that optimizing this loss can offer substantial gains in data efficiency. We apply our method to text classification tasks and show that it improves predictive accuracy over competing approaches on several benchmark datasets. We further evaluate our approach on the task of verifying large language model generations for correctness in grade-school math problems. Our results show that models trained with our method are better able to distinguish promising generations from unpromising ones after observing only a few tokens.