GAPNet: A Lightweight Framework for Image and Video Salient Object Detection via Granularity-Aware Paradigm

TL;DR

GAPNet is a lightweight, granularity-aware network for image and video salient object detection that achieves state-of-the-art performance with efficient feature fusion and supervision strategies.

Contribution

The paper introduces a novel granularity-aware paradigm with specialized modules for efficient multi-scale feature fusion in a lightweight SOD framework.

Findings

Achieves state-of-the-art results among lightweight SOD models

Uses granular pyramid convolution and cross-scale attention modules

Maintains high accuracy with negligible computational cost

Abstract

Recent salient object detection (SOD) models predominantly rely on heavyweight backbones, incurring substantial computational cost and hindering their practical application in various real-world settings, particularly on edge devices. This paper presents GAPNet, a lightweight network built on the granularity-aware paradigm for both image and video SOD. We assign saliency maps of different granularities to supervise the multi-scale decoder side-outputs: coarse object locations for high-level outputs and fine-grained object boundaries for low-level outputs. Specifically, our decoder is built with granularity-aware connections which fuse high-level features of low granularity and low-level features of high granularity, respectively. To support these connections, we design granular pyramid convolution (GPC) and cross-scale attention (CSA) modules for efficient fusion of low-scale and high-scale features, respectively. On top of the encoder, a self-attention module is built to learn global information, enabling accurate object localization with negligible computational cost. Unlike traditional U-Net-based approaches, our proposed method optimizes feature utilization and semantic interpretation while applying appropriate supervision at each processing stage. Extensive experiments show that the proposed method achieves a new state-of-the-art performance among lightweight image and video SOD models. Code is available at https://github.com/yuhuan-wu/GAPNet.