DM$^3$Net: Dual-Camera Super-Resolution via Domain Modulation and Multi-scale Matching

TL;DR

DM$^3$Net is a novel dual-camera super-resolution method that uses domain modulation and multi-scale matching to effectively enhance smartphone images, outperforming existing approaches.

Contribution

The paper introduces a dual-camera super-resolution network with domain modulation, multi-scale matching, and key pruning for improved accuracy and efficiency.

Findings

Outperforms state-of-the-art methods on real-world datasets.

Effectively bridges domain gap with global representations.

Reduces memory and inference time significantly.

Abstract

Dual-camera super-resolution is highly practical for smartphone photography that primarily super-resolve the wide-angle images using the telephoto image as a reference. In this paper, we propose DM$^3$Net, a novel dual-camera super-resolution network based on Domain Modulation and Multi-scale Matching. To bridge the domain gap between the high-resolution domain and the degraded domain, we learn two compressed global representations from image pairs corresponding to the two domains. To enable reliable transfer of high-frequency structural details from the reference image, we design a multi-scale matching module that conducts patch-level feature matching and retrieval across multiple receptive fields to improve matching accuracy and robustness. Moreover, we also introduce Key Pruning to achieve a significant reduction in memory usage and inference time with little model performance sacrificed. Experimental results on three real-world datasets demonstrate that our DM$^3$Net outperforms the state-of-the-art approaches.