Learning Parallax Transformer Network for Stereo Image JPEG Artifacts Removal

TL;DR

This paper introduces PTNet, a novel stereo image JPEG artifacts removal network that leverages a parallax transformer and confidence-based fusion to effectively utilize stereo view information, outperforming existing methods.

Contribution

The paper proposes a parallax transformer network with a bi-directional module and confidence-based fusion for stereo JPEG artifacts removal, addressing view alignment challenges.

Findings

PTNet outperforms state-of-the-art methods in artifact removal.

The coarse-to-fine cross-view interaction improves performance.

The confidence-based fusion enhances feature integration across views.

Abstract

Under stereo settings, the performance of image JPEG artifacts removal can be further improved by exploiting the additional information provided by a second view. However, incorporating this information for stereo image JPEG artifacts removal is a huge challenge, since the existing compression artifacts make pixel-level view alignment difficult. In this paper, we propose a novel parallax transformer network (PTNet) to integrate the information from stereo image pairs for stereo image JPEG artifacts removal. Specifically, a well-designed symmetric bi-directional parallax transformer module is proposed to match features with similar textures between different views instead of pixel-level view alignment. Due to the issues of occlusions and boundaries, a confidence-based cross-view fusion module is proposed to achieve better feature fusion for both views, where the cross-view features are weighted with confidence maps. Especially, we adopt a coarse-to-fine design for the cross-view interaction, leading to better performance. Comprehensive experimental results demonstrate that our PTNet can effectively remove compression artifacts and achieves superior performance than other testing state-of-the-art methods.