FTIO: Frequent Temporally Integrated Objects

TL;DR

FTIO is a post-processing framework that enhances unsupervised video object segmentation by improving object selection and correcting temporal inconsistencies, achieving state-of-the-art results.

Contribution

It introduces a combined criterion for better object selection and a three-stage method to correct temporal inconsistencies in UVOS.

Findings

Achieves state-of-the-art performance in multi-object UVOS

Effectively mitigates failures with small or complex objects

Improves temporal consistency in object segmentation

Abstract

Predicting and tracking objects in real-world scenarios is a critical challenge in Video Object Segmentation (VOS) tasks. Unsupervised VOS (UVOS) has the additional challenge of finding an initial segmentation of salient objects, which affects the entire process and keeps a permanent uncertainty about the object proposals. Moreover, deformation and fast motion can lead to temporal inconsistencies. To address these problems, we propose Frequent Temporally Integrated Objects (FTIO), a post-processing framework with two key components. First, we introduce a combined criterion to improve object selection, mitigating failures common in UVOS--particularly when objects are small or structurally complex--by extracting frequently appearing salient objects. Second, we present a three-stage method to correct temporal inconsistencies by integrating missing object mask regions. Experimental results demonstrate that FTIO achieves state-of-the-art performance in multi-object UVOS. Code is available at: https://github.com/MohammadMohammadzadehKalati/FTIO