TENet: Triple Excitation Network for Video Salient Object Detection

TL;DR

This paper introduces the Triple Excitation Network, a novel approach for video salient object detection that enhances training with spatial, temporal, and online excitations, leading to improved accuracy and convergence.

Contribution

The paper presents a new triple excitation mechanism and an online refinement strategy for VSOD, advancing training techniques and achieving state-of-the-art results.

Findings

Outperforms existing VSOD methods in accuracy.

Effective reduction of saliency shifting issues.

Enables online saliency boosting during testing.

Abstract

In this paper, we propose a simple yet effective approach, named Triple Excitation Network, to reinforce the training of video salient object detection (VSOD) from three aspects, spatial, temporal, and online excitations. These excitation mechanisms are designed following the spirit of curriculum learning and aim to reduce learning ambiguities at the beginning of training by selectively exciting feature activations using ground truth. Then we gradually reduce the weight of ground truth excitations by a curriculum rate and replace it by a curriculum complementary map for better and faster convergence. In particular, the spatial excitation strengthens feature activations for clear object boundaries, while the temporal excitation imposes motions to emphasize spatio-temporal salient regions. Spatial and temporal excitations can combat the saliency shifting problem and conflict between spatial and temporal features of VSOD. Furthermore, our semi-curriculum learning design enables the first online refinement strategy for VSOD, which allows exciting and boosting saliency responses during testing without re-training. The proposed triple excitations can easily plug in different VSOD methods. Extensive experiments show the effectiveness of all three excitation methods and the proposed method outperforms state-of-the-art image and video salient object detection methods.