SelfReformer: Self-Refined Network with Transformer for Salient Object Detection

TL;DR

SelfReformer is a Transformer-based network that improves salient object detection by explicitly learning global context and refining local details, achieving state-of-the-art results on multiple benchmarks.

Contribution

The paper introduces a novel SelfReformer network that combines Transformer-based global context learning with a Pixel Shuffle-based upsampling and a two-stage refinement module for superior SOD performance.

Findings

Achieves state-of-the-art performance on five benchmark datasets.

Effectively captures long-range dependencies with Transformer architecture.

Refines local details through a two-stage context refinement module.

Abstract

The global and local contexts significantly contribute to the integrity of predictions in Salient Object Detection (SOD). Unfortunately, existing methods still struggle to generate complete predictions with fine details. There are two major problems in conventional approaches: first, for global context, high-level CNN-based encoder features cannot effectively catch long-range dependencies, resulting in incomplete predictions. Second, downsampling the ground truth to fit the size of predictions will introduce inaccuracy as the ground truth details are lost during interpolation or pooling. Thus, in this work, we developed a Transformer-based network and framed a supervised task for a branch to learn the global context information explicitly. Besides, we adopt Pixel Shuffle from Super-Resolution (SR) to reshape the predictions back to the size of ground truth instead of the reverse. Thus details in the ground truth are untouched. In addition, we developed a two-stage Context Refinement Module (CRM) to fuse global context and automatically locate and refine the local details in the predictions. The proposed network can guide and correct itself based on the global and local context generated, thus is named, Self-Refined Transformer (SelfReformer). Extensive experiments and evaluation results on five benchmark datasets demonstrate the outstanding performance of the network, and we achieved the state-of-the-art.