ResFlow: Fine-tuning Residual Optical Flow for Event-based High Temporal Resolution Motion Estimation

TL;DR

ResFlow introduces a residual-based method for high-temporal-resolution optical flow estimation from event data, effectively addressing data sparsity and lack of ground truth, and achieves state-of-the-art results.

Contribution

The paper proposes a novel residual paradigm and learning strategies for event-based HTR optical flow, improving accuracy and robustness over existing methods.

Findings

Achieves state-of-the-art accuracy in HTR and LTR optical flow estimation.

Effectively mitigates event sparsity impacts on optimization.

Supports in-domain self-supervised training.

Abstract

Event cameras hold significant promise for high-temporal-resolution (HTR) motion estimation. However, estimating event-based HTR optical flow faces two key challenges: the absence of HTR ground-truth data and the intrinsic sparsity of event data. Most existing approaches rely on the flow accumulation paradigms to indirectly supervise intermediate flows, often resulting in accumulation errors and optimization difficulties. To address these challenges, we propose a residual-based paradigm for estimating HTR optical flow with event data. Our approach separates HTR flow estimation into two stages: global linear motion estimation and HTR residual flow refinement. The residual paradigm effectively mitigates the impacts of event sparsity on optimization and is compatible with any LTR algorithm. Next, to address the challenge posed by the absence of HTR ground truth, we incorporate novel learning strategies. Specifically, we initially employ a shared refiner to estimate the residual flows, enabling both LTR supervision and HTR inference. Subsequently, we introduce regional noise to simulate the residual patterns of intermediate flows, facilitating the adaptation from LTR supervision to HTR inference. Additionally, we show that the noise-based strategy supports in-domain self-supervised training. Comprehensive experimental results demonstrate that our approach achieves state-of-the-art accuracy in both LTR and HTR metrics, highlighting its effectiveness and superiority.