InterpoNet, A brain inspired neural network for optical flow dense interpolation

TL;DR

InterpoNet is a brain-inspired neural network for optical flow interpolation that leverages lateral dependencies and multi-layer supervision, outperforming traditional methods like EpicFlow on key benchmarks.

Contribution

A novel data-driven, fully convolutional network for optical flow interpolation inspired by visual cortex processes, incorporating lateral dependencies and contour information.

Findings

Outperforms EpicFlow on benchmarks

Achieves state-of-the-art results on Sintel and KITTI 2012

Robust across various matching algorithms

Abstract

Sparse-to-dense interpolation for optical flow is a fundamental phase in the pipeline of most of the leading optical flow estimation algorithms. The current state-of-the-art method for interpolation, EpicFlow, is a local average method based on an edge aware geodesic distance. We propose a new data-driven sparse-to-dense interpolation algorithm based on a fully convolutional network. We draw inspiration from the filling-in process in the visual cortex and introduce lateral dependencies between neurons and multi-layer supervision into our learning process. We also show the importance of the image contour to the learning process. Our method is robust and outperforms EpicFlow on competitive optical flow benchmarks with several underlying matching algorithms. This leads to state-of-the-art performance on the Sintel and KITTI 2012 benchmarks.