FOMTrace: Interactive Video Segmentation By Image Graphs and Fuzzy Object Models

TL;DR

FOMTrace is an interactive video segmentation method that uses image graphs and fuzzy object models to efficiently trace objects through video frames with user-guided refinement.

Contribution

It introduces a novel approach combining semi-automatic segmentation with fuzzy object models for real-time interactive video object tracing.

Findings

Outperforms state-of-the-art interactive segmentation methods

Handles fast-moving and deformable objects effectively

Provides real-time response for user interactions

Abstract

Common users have changed from mere consumers to active producers of multimedia data content. Video editing plays an important role in this scenario, calling for simple segmentation tools that can handle fast-moving and deformable video objects with possible occlusions, color similarities with the background, among other challenges. We present an interactive video segmentation method, named FOMTrace, which addresses the problem in an effective and efficient way. From a user-provided object mask in a first frame, the method performs semi-automatic video segmentation on a spatiotemporal superpixel-graph, and then estimates a Fuzzy Object Model (FOM), which refines segmentation of the second frame by constraining delineation on a pixel-graph within a region where the object's boundary is expected to be. The user can correct/accept the refined object mask in the second frame, which is then similarly used to improve the spatiotemporal video segmentation of the remaining frames. Both steps are repeated alternately, within interactive response times, until the segmentation refinement of the final frame is accepted by the user. Extensive experiments demonstrate FOMTrace's ability for tracing objects in comparison with state-of-the-art approaches for interactive video segmentation, supervised, and unsupervised object tracking.