Unsupervised Flow Refinement near Motion Boundaries

TL;DR

This paper introduces an unsupervised method to refine optical flow near motion boundaries by detecting boundaries and replacing motions, significantly improving flow accuracy in these challenging regions without extra training.

Contribution

It proposes a boundary detection-based refinement framework that enhances unsupervised optical flow estimates near motion boundaries, applicable to any predictor.

Findings

Improved boundary detection accuracy over baseline methods.

Enhanced flow estimates near motion boundaries without additional training.

Compatible with various existing flow predictors.

Abstract

Unsupervised optical flow estimators based on deep learning have attracted increasing attention due to the cost and difficulty of annotating for ground truth. Although performance measured by average End-Point Error (EPE) has improved over the years, flow estimates are still poorer along motion boundaries (MBs), where the flow is not smooth, as is typically assumed, and where features computed by neural networks are contaminated by multiple motions. To improve flow in the unsupervised settings, we design a framework that detects MBs by analyzing visual changes along boundary candidates and replaces motions close to detections with motions farther away. Our proposed algorithm detects boundaries more accurately than a baseline method with the same inputs and can improve estimates from any flow predictor without additional training.