Fast Template Matching and Update for Video Object Tracking and Segmentation

TL;DR

This paper introduces a reinforcement learning-based approach for video object segmentation that improves matching and template updating decisions, achieving near 10x speedup and higher accuracy over previous methods.

Contribution

It presents a novel reinforcement learning framework to optimize matching and updating decisions in video object segmentation, enhancing speed and accuracy.

Findings

Achieves 69.1% region similarity on DAVIS 2017

Nearly 10 times faster than previous state-of-the-art

Improves decision-making in template updating and matching

Abstract

In this paper, the main task we aim to tackle is the multi-instance semi-supervised video object segmentation across a sequence of frames where only the first-frame box-level ground-truth is provided. Detection-based algorithms are widely adopted to handle this task, and the challenges lie in the selection of the matching method to predict the result as well as to decide whether to update the target template using the newly predicted result. The existing methods, however, make these selections in a rough and inflexible way, compromising their performance. To overcome this limitation, we propose a novel approach which utilizes reinforcement learning to make these two decisions at the same time. Specifically, the reinforcement learning agent learns to decide whether to update the target template according to the quality of the predicted result. The choice of the matching method will be determined at the same time, based on the action history of the reinforcement learning agent. Experiments show that our method is almost 10 times faster than the previous state-of-the-art method with even higher accuracy (region similarity of 69.1% on DAVIS 2017 dataset).