Recurrent Structure Attention Guidance for Depth Super-Resolution

TL;DR

This paper introduces a recurrent structure attention framework for depth super-resolution that adaptively separates high-frequency details and iteratively refines guidance using image features, outperforming existing methods.

Contribution

The proposed method combines a contrastive network for adaptive frequency separation with a recurrent attention mechanism for refined guidance, advancing depth super-resolution techniques.

Findings

Achieves superior performance over state-of-the-art methods

Effectively separates high-frequency components adaptively

Refines guidance iteratively for improved depth recovery

Abstract

Image guidance is an effective strategy for depth super-resolution. Generally, most existing methods employ hand-crafted operators to decompose the high-frequency (HF) and low-frequency (LF) ingredients from low-resolution depth maps and guide the HF ingredients by directly concatenating them with image features. However, the hand-designed operators usually cause inferior HF maps (e.g., distorted or structurally missing) due to the diverse appearance of complex depth maps. Moreover, the direct concatenation often results in weak guidance because not all image features have a positive effect on the HF maps. In this paper, we develop a recurrent structure attention guided (RSAG) framework, consisting of two important parts. First, we introduce a deep contrastive network with multi-scale filters for adaptive frequency-domain separation, which adopts contrastive networks from large filters to small ones to calculate the pixel contrasts for adaptive high-quality HF predictions. Second, instead of the coarse concatenation guidance, we propose a recurrent structure attention block, which iteratively utilizes the latest depth estimation and the image features to jointly select clear patterns and boundaries, aiming at providing refined guidance for accurate depth recovery. In addition, we fuse the features of HF maps to enhance the edge structures in the decomposed LF maps. Extensive experiments show that our approach obtains superior performance compared with state-of-the-art depth super-resolution methods.