Learning to Anticipate Future with Dynamic Context Removal

TL;DR

This paper introduces Dynamic Context Removal, a novel training scheme for anticipation models that gradually increases difficulty by removing context, leading to state-of-the-art results across multiple benchmarks.

Contribution

It proposes a human-like curriculum learning approach for training anticipation models, enhancing effectiveness and efficiency, and is compatible with various reasoning architectures.

Findings

Achieves state-of-the-art performance on four benchmarks.

Compatible with transformer and LSTM models.

Demonstrates improved anticipation accuracy and training efficiency.

Abstract

Anticipating future events is an essential feature for intelligent systems and embodied AI. However, compared to the traditional recognition task, the uncertainty of future and reasoning ability requirement make the anticipation task very challenging and far beyond solved. In this filed, previous methods usually care more about the model architecture design or but few attention has been put on how to train an anticipation model with a proper learning policy. To this end, in this work, we propose a novel training scheme called Dynamic Context Removal (DCR), which dynamically schedules the visibility of observed future in the learning procedure. It follows the human-like curriculum learning process, i.e., gradually removing the event context to increase the anticipation difficulty till satisfying the final anticipation target. Our learning scheme is plug-and-play and easy to integrate any reasoning model including transformer and LSTM, with advantages in both effectiveness and efficiency. In extensive experiments, the proposed method achieves state-of-the-art on four widely-used benchmarks. Our code and models are publicly released at https://github.com/AllenXuuu/DCR.