Soft-Landing Strategy for Alleviating the Task Discrepancy Problem in Temporal Action Localization Tasks

TL;DR

This paper introduces the Soft-Landing (SoLa) strategy, a lightweight framework that improves temporal action localization by bridging the gap between pretrained feature encoders and downstream tasks without heavy retraining.

Contribution

The paper proposes a novel SoLa module and an unsupervised training scheme that enhances transferability in TAL tasks efficiently and without requiring temporal annotations.

Findings

Effective in reducing task discrepancy in TAL

Achieves high accuracy with low computational cost

Validated on multiple benchmarks

Abstract

Temporal Action Localization (TAL) methods typically operate on top of feature sequences from a frozen snippet encoder that is pretrained with the Trimmed Action Classification (TAC) tasks, resulting in a task discrepancy problem. While existing TAL methods mitigate this issue either by retraining the encoder with a pretext task or by end-to-end fine-tuning, they commonly require an overload of high memory and computation. In this work, we introduce Soft-Landing (SoLa) strategy, an efficient yet effective framework to bridge the transferability gap between the pretrained encoder and the downstream tasks by incorporating a light-weight neural network, i.e., a SoLa module, on top of the frozen encoder. We also propose an unsupervised training scheme for the SoLa module; it learns with inter-frame Similarity Matching that uses the frame interval as its supervisory signal, eliminating the need for temporal annotations. Experimental evaluation on various benchmarks for downstream TAL tasks shows that our method effectively alleviates the task discrepancy problem with remarkable computational efficiency.