Rapid Salient Object Detection with Difference Convolutional Neural Networks

TL;DR

This paper introduces efficient CNN-based models for real-time salient object detection on resource-limited devices, utilizing novel difference convolution techniques to improve speed and accuracy for images and videos.

Contribution

The paper proposes Pixel Difference Convolutions and a reparameterization strategy to embed contrast cues into CNNs, enabling fast and accurate salient object detection on embedded devices.

Findings

Models operate at 46 FPS and 150 FPS on Jetson Orin with less than 1M parameters.

Our models outperform lightweight competitors by over 2x and 3x in speed.

Achieve superior accuracy in real-time image and video SOD tasks.

Abstract

This paper addresses the challenge of deploying salient object detection (SOD) on resource-constrained devices with real-time performance. While recent advances in deep neural networks have improved SOD, existing top-leading models are computationally expensive. We propose an efficient network design that combines traditional wisdom on SOD and the representation power of modern CNNs. Like biologically-inspired classical SOD methods relying on computing contrast cues to determine saliency of image regions, our model leverages Pixel Difference Convolutions (PDCs) to encode the feature contrasts. Differently, PDCs are incorporated in a CNN architecture so that the valuable contrast cues are extracted from rich feature maps. For efficiency, we introduce a difference convolution reparameterization (DCR) strategy that embeds PDCs into standard convolutions, eliminating computation and parameters at inference. Additionally, we introduce SpatioTemporal Difference Convolution (STDC) for video SOD, enhancing the standard 3D convolution with spatiotemporal contrast capture. Our models, SDNet for image SOD and STDNet for video SOD, achieve significant improvements in efficiency-accuracy trade-offs. On a Jetson Orin device, our models with $<$ 1M parameters operate at 46 FPS and 150 FPS on streamed images and videos, surpassing the second-best lightweight models in our experiments by more than $2\times$ and $3\times$ in speed with superior accuracy. Code will be available at https://github.com/hellozhuo/stdnet.git.