Layered Optical Flow Estimation Using a Deep Neural Network with a Soft Mask

TL;DR

This paper introduces a deep learning method that automatically generates layered optical flow representations using a soft-mask module, improving accuracy in flow estimation especially around discontinuities and occlusions.

Contribution

It proposes a novel soft-mask module that can be integrated into existing networks like FlowNet to enhance layered optical flow estimation without pre-segmentation.

Findings

Achieves better flow estimation results than original FlowNet

Effective handling of motion discontinuities and occlusions

Applicable to both supervised and unsupervised tasks

Abstract

Using a layered representation for motion estimation has the advantage of being able to cope with discontinuities and occlusions. In this paper, we learn to estimate optical flow by combining a layered motion representation with deep learning. Instead of pre-segmenting the image to layers, the proposed approach automatically generates a layered representation of optical flow using the proposed soft-mask module. The essential components of the soft-mask module are maxout and fuse operations, which enable a disjoint layered representation of optical flow and more accurate flow estimation. We show that by using masks the motion estimate results in a quadratic function of input features in the output layer. The proposed soft-mask module can be added to any existing optical flow estimation networks by replacing their flow output layer. In this work, we use FlowNet as the base network to which we add the soft-mask module. The resulting network is tested on three well-known benchmarks with both supervised and unsupervised flow estimation tasks. Evaluation results show that the proposed network achieve better results compared with the original FlowNet.