VST++: Efficient and Stronger Visual Saliency Transformer

TL;DR

VST++ is an improved transformer-based model for salient object detection that enhances efficiency and accuracy by introducing novel modules and encoding methods, outperforming existing methods with reduced computational costs.

Contribution

The paper presents VST++, a more efficient and stronger version of VST, featuring a novel attention module, depth encoding, and token supervision to improve salient object detection performance.

Findings

Outperforms existing methods on RGB, RGB-D, and RGB-T benchmarks.

Reduces computational costs by 25% compared to VST.

Demonstrates strong generalization and improved accuracy.

Abstract

While previous CNN-based models have exhibited promising results for salient object detection (SOD), their ability to explore global long-range dependencies is restricted. Our previous work, the Visual Saliency Transformer (VST), addressed this constraint from a transformer-based sequence-to-sequence perspective, to unify RGB and RGB-D SOD. In VST, we developed a multi-task transformer decoder that concurrently predicts saliency and boundary outcomes in a pure transformer architecture. Moreover, we introduced a novel token upsampling method called reverse T2T for predicting a high-resolution saliency map effortlessly within transformer-based structures. Building upon the VST model, we further propose an efficient and stronger VST version in this work, i.e. VST++. To mitigate the computational costs of the VST model, we propose a Select-Integrate Attention (SIA) module, partitioning foreground into fine-grained segments and aggregating background information into a single coarse-grained token. To incorporate 3D depth information with low cost, we design a novel depth position encoding method tailored for depth maps. Furthermore, we introduce a token-supervised prediction loss to provide straightforward guidance for the task-related tokens. We evaluate our VST++ model across various transformer-based backbones on RGB, RGB-D, and RGB-T SOD benchmark datasets. Experimental results show that our model outperforms existing methods while achieving a 25% reduction in computational costs without significant performance compromise. The demonstrated strong ability for generalization, enhanced performance, and heightened efficiency of our VST++ model highlight its potential.