Cutting-Edge Techniques for Depth Map Super-Resolution

TL;DR

This paper introduces a novel depth map super-resolution method using Swin transformer architecture and a NAF CNN, achieving improved performance over existing techniques while maintaining efficiency.

Contribution

It presents a new joint image filtering DMSR algorithm with Swin transformer and compares it with a NAF CNN, advancing state-of-the-art in depth map super-resolution.

Findings

Improved super-resolution quality over state-of-the-art methods.

Maintains competitive computation time for noisy depth maps.

Demonstrates effectiveness through numerical and visual validation.

Abstract

To overcome hardware limitations in commercially available depth sensors which result in low-resolution depth maps, depth map super-resolution (DMSR) is a practical and valuable computer vision task. DMSR requires upscaling a low-resolution (LR) depth map into a high-resolution (HR) space. Joint image filtering for DMSR has been applied using spatially-invariant and spatially-variant convolutional neural network (CNN) approaches. In this project, we propose a novel joint image filtering DMSR algorithm using a Swin transformer architecture. Furthermore, we introduce a Nonlinear Activation Free (NAF) network based on a conventional CNN model used in cutting-edge image restoration applications and compare the performance of the techniques. The proposed algorithms are validated through numerical studies and visual examples demonstrating improvements to state-of-the-art performance while maintaining competitive computation time for noisy depth map super-resolution.