TemporalMaxer: Maximize Temporal Context with only Max Pooling for Temporal Action Localization

TL;DR

TemporalMaxer offers a simple, parameter-free max-pooling approach to enhance temporal action localization by effectively capturing critical local information, outperforming complex models with fewer resources.

Contribution

This paper introduces TemporalMaxer, a minimalistic max-pooling method that replaces complex long-term modeling, achieving superior performance with less computational cost.

Findings

Outperforms state-of-the-art methods on multiple datasets

Requires significantly fewer parameters and computational resources

Effectively captures critical local temporal information

Abstract

Temporal Action Localization (TAL) is a challenging task in video understanding that aims to identify and localize actions within a video sequence. Recent studies have emphasized the importance of applying long-term temporal context modeling (TCM) blocks to the extracted video clip features such as employing complex self-attention mechanisms. In this paper, we present the simplest method ever to address this task and argue that the extracted video clip features are already informative to achieve outstanding performance without sophisticated architectures. To this end, we introduce TemporalMaxer, which minimizes long-term temporal context modeling while maximizing information from the extracted video clip features with a basic, parameter-free, and local region operating max-pooling block. Picking out only the most critical information for adjacent and local clip embeddings, this block results in a more efficient TAL model. We demonstrate that TemporalMaxer outperforms other state-of-the-art methods that utilize long-term TCM such as self-attention on various TAL datasets while requiring significantly fewer parameters and computational resources. The code for our approach is publicly available at https://github.com/TuanTNG/TemporalMaxer