Learning to Stop Overthinking at Test Time

TL;DR

This paper introduces a test time training method and a new recurrent architecture, Conv-LiGRU, to optimize computation per sample, reduce overthinking, and improve accuracy in visual reasoning tasks.

Contribution

It proposes a novel test time training approach and the Conv-LiGRU architecture to dynamically determine optimal computation, addressing overthinking in deep-thinking models.

Findings

Conv-LiGRU outperforms DT in stability and accuracy.

The method effectively mitigates overthinking during test time.

Experiments show improved efficiency and robustness.

Abstract

Test time scaling is currently one of the most active research areas that shows promise after training time scaling has reached its limits. Deep-thinking (DT) models are a class of recurrent models that can perform easy-to-hard generalization by assigning more compute to harder test samples. However, due to their inability to determine the complexity of a test sample, DT models have to use a large amount of computation for both easy and hard test samples. Excessive test time computation is wasteful and can cause the ``overthinking'' problem where more test time computation leads to worse results. In this paper, we introduce a test time training method for determining the optimal amount of computation needed for each sample during test time. We also propose Conv-LiGRU, a novel recurrent architecture for efficient and robust visual reasoning. Extensive experiments demonstrate that Conv-LiGRU is more stable than DT, effectively mitigates the ``overthinking'' phenomenon, and achieves superior accuracy.