Shift Convolution Network for Stereo Matching

TL;DR

This paper introduces ShiftConvNet, a fast and accurate stereo matching network that replaces traditional correlation with shift convolution layers, achieving state-of-the-art results on benchmark datasets.

Contribution

The paper proposes a novel shift convolution layer and architecture for stereo matching, improving speed and accuracy over existing methods.

Findings

Achieves state-of-the-art results on FlyingThings 3D dataset.

Runs at 5 fps, faster than comparable methods.

Improves disparity estimation accuracy with auto shift convolution refinement.

Abstract

In this paper, we present Shift Convolution Network (ShiftConvNet) to provide matching capability between two feature maps for stereo estimation. The proposed method can speedily produce a highly accurate disparity map from stereo images. A module called shift convolution layer is proposed to replace the traditional correlation layer to perform patch comparisons between two feature maps. By using a novel architecture of convolutional network to learn the matching process, ShiftConvNet can produce better results than DispNet-C[1], also running faster with 5 fps. Moreover, with a proposed auto shift convolution refine part, further improvement is obtained. The proposed approach was evaluated on FlyingThings 3D. It achieves state-of-the-art results on the benchmark dataset. Codes will be made available at github.