Learning Dynamic Network Using a Reuse Gate Function in Semi-supervised Video Object Segmentation

TL;DR

This paper introduces a dynamic network with a reuse gate function for semi-supervised video object segmentation, significantly improving inference speed by skipping unnecessary computations for stationary or slow-moving objects while maintaining high segmentation quality.

Contribution

It proposes a novel change estimation and reuse mechanism that adaptively skips full network processing, enhancing efficiency in semi-supervised video object segmentation.

Findings

Significant speedup in inference without much accuracy loss.

Effective across multiple Semi-VOS datasets like DAVIS and YouTube-VOS.

Demonstrates general applicability to various Semi-VOS methods.

Abstract

Current state-of-the-art approaches for Semi-supervised Video Object Segmentation (Semi-VOS) propagates information from previous frames to generate segmentation mask for the current frame. This results in high-quality segmentation across challenging scenarios such as changes in appearance and occlusion. But it also leads to unnecessary computations for stationary or slow-moving objects where the change across frames is minimal. In this work, we exploit this observation by using temporal information to quickly identify frames with minimal change and skip the heavyweight mask generation step. To realize this efficiency, we propose a novel dynamic network that estimates change across frames and decides which path -- computing a full network or reusing previous frame's feature -- to choose depending on the expected similarity. Experimental results show that our approach significantly improves inference speed without much accuracy degradation on challenging Semi-VOS datasets -- DAVIS 16, DAVIS 17, and YouTube-VOS. Furthermore, our approach can be applied to multiple Semi-VOS methods demonstrating its generality. The code is available in https://github.com/HYOJINPARK/Reuse_VOS.