SUNet: Symmetric Undistortion Network for Rolling Shutter Correction

TL;DR

This paper introduces SUNet, a deep neural network that corrects rolling shutter distortions by symmetrically estimating a global shutter image from two consecutive frames, outperforming existing methods.

Contribution

The paper proposes a novel symmetric deep network architecture for rolling shutter correction using two frames, with a unique consistency constraint for improved accuracy.

Findings

Outperforms state-of-the-art methods on synthetic and real datasets.

Effectively estimates high-quality global shutter images from rolling shutter inputs.

Demonstrates robustness across various camera motions and scenes.

Abstract

The vast majority of modern consumer-grade cameras employ a rolling shutter mechanism, leading to image distortions if the camera moves during image acquisition. In this paper, we present a novel deep network to solve the generic rolling shutter correction problem with two consecutive frames. Our pipeline is symmetrically designed to predict the global shutter image corresponding to the intermediate time of these two frames, which is difficult for existing methods because it corresponds to a camera pose that differs most from the two frames. First, two time-symmetric dense undistortion flows are estimated by using well-established principles: pyramidal construction, warping, and cost volume processing. Then, both rolling shutter images are warped into a common global shutter one in the feature space, respectively. Finally, a symmetric consistency constraint is constructed in the image decoder to effectively aggregate the contextual cues of two rolling shutter images, thereby recovering the high-quality global shutter image. Extensive experiments with both synthetic and real data from public benchmarks demonstrate the superiority of our proposed approach over the state-of-the-art methods.