Uncertainty-boosted Robust Video Activity Anticipation

TL;DR

This paper introduces a novel framework for video activity anticipation that incorporates uncertainty estimation to improve robustness and interpretability, addressing the challenge of data uncertainty in dynamic video content.

Contribution

It proposes an uncertainty-boosted approach that models and utilizes uncertainty values to enhance prediction reliability and model generalization in video activity anticipation.

Findings

Achieves improved performance across multiple benchmarks.

Demonstrates enhanced robustness and interpretability.

Effective uncertainty modeling through relative uncertainty comparison.

Abstract

Video activity anticipation aims to predict what will happen in the future, embracing a broad application prospect ranging from robot vision and autonomous driving. Despite the recent progress, the data uncertainty issue, reflected as the content evolution process and dynamic correlation in event labels, has been somehow ignored. This reduces the model generalization ability and deep understanding on video content, leading to serious error accumulation and degraded performance. In this paper, we address the uncertainty learning problem and propose an uncertainty-boosted robust video activity anticipation framework, which generates uncertainty values to indicate the credibility of the anticipation results. The uncertainty value is used to derive a temperature parameter in the softmax function to modulate the predicted target activity distribution. To guarantee the distribution adjustment, we construct a reasonable target activity label representation by incorporating the activity evolution from the temporal class correlation and the semantic relationship. Moreover, we quantify the uncertainty into relative values by comparing the uncertainty among sample pairs and their temporal-lengths. This relative strategy provides a more accessible way in uncertainty modeling than quantifying the absolute uncertainty values on the whole dataset. Experiments on multiple backbones and benchmarks show our framework achieves promising performance and better robustness/interpretability. Source codes are available at https://github.com/qzhb/UbRV2A.