Drift Robust Non-rigid Optical Flow Enhancement for Long Sequences

TL;DR

This paper presents a new optimization framework with an Anchor Patch constraint that significantly reduces drift errors in long-term dense tracking of nonrigid objects, applicable to various optical flow algorithms.

Contribution

It introduces a novel Anchor Patch constraint framework that enhances long sequence nonrigid optical flow tracking, reducing accumulated errors.

Findings

Significant error reduction on 6 optical flow algorithms.

Effective in real-world nonrigid benchmarks.

Robust against synthetic occlusions and noise.

Abstract

It is hard to densely track a nonrigid object in long term, which is a fundamental research issue in the computer vision community. This task often relies on estimating pairwise correspondences between images over time where the error is accumulated and leads to a drift issue. In this paper, we introduce a novel optimization framework with an Anchor Patch constraint. It is supposed to significantly reduce overall errors given long sequences containing non-rigidly deformable objects. Our framework can be applied to any dense tracking algorithm, e.g. optical flow. We demonstrate the success of our approach by showing significant error reduction on 6 popular optical flow algorithms applied to a range of real-world nonrigid benchmarks. We also provide quantitative analysis of our approach given synthetic occlusions and image noise.