Implicit Motion Handling for Video Camouflaged Object Detection

TL;DR

This paper introduces a novel video camouflaged object detection framework that jointly models motion and segmentation, leveraging implicit motion estimation and a spatio-temporal transformer to improve detection accuracy in videos.

Contribution

It unifies motion estimation and segmentation in a single framework using dense correlation volumes and employs a spatio-temporal transformer for temporal consistency, advancing VCOD methods.

Findings

Outperforms existing VCOD methods on benchmarks

Introduces a large-scale VCOD dataset MoCA-Mask

Demonstrates the effectiveness of implicit motion modeling

Abstract

We propose a new video camouflaged object detection (VCOD) framework that can exploit both short-term dynamics and long-term temporal consistency to detect camouflaged objects from video frames. An essential property of camouflaged objects is that they usually exhibit patterns similar to the background and thus make them hard to identify from still images. Therefore, effectively handling temporal dynamics in videos becomes the key for the VCOD task as the camouflaged objects will be noticeable when they move. However, current VCOD methods often leverage homography or optical flows to represent motions, where the detection error may accumulate from both the motion estimation error and the segmentation error. On the other hand, our method unifies motion estimation and object segmentation within a single optimization framework. Specifically, we build a dense correlation volume to implicitly capture motions between neighbouring frames and utilize the final segmentation supervision to optimize the implicit motion estimation and segmentation jointly. Furthermore, to enforce temporal consistency within a video sequence, we jointly utilize a spatio-temporal transformer to refine the short-term predictions. Extensive experiments on VCOD benchmarks demonstrate the architectural effectiveness of our approach. We also provide a large-scale VCOD dataset named MoCA-Mask with pixel-level handcrafted ground-truth masks and construct a comprehensive VCOD benchmark with previous methods to facilitate research in this direction. Dataset Link: https://xueliancheng.github.io/SLT-Net-project.