Finding Correspondences for Optical Flow and Disparity Estimations using a Sub-pixel Convolution-based Encoder-Decoder Network

TL;DR

This paper introduces a sub-pixel convolution-based encoder-decoder network that improves optical flow and disparity estimation accuracy over existing models like FlowNetS and DispNet by replacing deconvolution layers with sub-pixel convolution blocks.

Contribution

The novel network architecture enhances optical flow and disparity estimation accuracy while reducing parameters, using sub-pixel convolution for refinement instead of traditional deconvolution.

Findings

Outperforms FlowNetS and DispNet in accuracy

Reduces training time and model complexity

Effective for both supervised and unsupervised tasks

Abstract

Deep convolutional neural networks (DCNN) have recently shown promising results in low-level computer vision problems such as optical flow and disparity estimation, but still, have much room to further improve their performance. In this paper, we propose a novel sub-pixel convolution-based encoder-decoder network for optical flow and disparity estimations, which can extend FlowNetS and DispNet by replacing the deconvolution layers with sup-pixel convolution blocks. By using sub-pixel refinement and estimation on the decoder stages instead of deconvolution, we can significantly improve the estimation accuracy for optical flow and disparity, even with reduced numbers of parameters. We show a supervised end-to-end training of our proposed networks for optical flow and disparity estimations, and an unsupervised end-to-end training for monocular depth and pose estimations. In order to verify the effectiveness of our proposed networks, we perform intensive experiments for (i) optical flow and disparity estimations, and (ii) monocular depth and pose estimations. Throughout the extensive experiments, our proposed networks outperform the baselines such as FlowNetS and DispNet in terms of estimation accuracy and training times.