Adaptive ROI Generation for Video Object Segmentation Using Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning-based method for adaptive region of interest selection in semi-supervised video object segmentation, significantly improving accuracy and speed over existing approaches.

Contribution

It proposes a novel RL framework with a multi-branch tree exploration method for optimal ROI selection, enhancing online model adaptation in video segmentation.

Findings

Achieves 87.1% mean region similarity on DAVIS 2016

Outperforms state-of-the-art methods in segmentation accuracy

Speeds up model adaptation process

Abstract

In this paper, we aim to tackle the task of semi-supervised video object segmentation across a sequence of frames where only the ground-truth segmentation of the first frame is provided. The challenges lie in how to online update the segmentation model initialized from the first frame adaptively and accurately, even in presence of multiple confusing instances or large object motion. The existing approaches rely on selecting the region of interest for model update, which however, is rough and inflexible, leading to performance degradation. To overcome this limitation, we propose a novel approach which utilizes reinforcement learning to select optimal adaptation areas for each frame, based on the historical segmentation information. The RL model learns to take optimal actions to adjust the region of interest inferred from the previous frame for online model updating. To speed up the model adaption, we further design a novel multi-branch tree based exploration method to fast select the best state action pairs. Our experiments show that our work improves the state-of-the-art of the mean region similarity on DAVIS 2016 dataset to 87.1%.