Interpretable Deep Multimodal Image Super-Resolution
Iman Marivani, Evaggelia Tsiligianni, Bruno Cornelis, Nikos, Deligiannis

TL;DR
This paper introduces an interpretable deep multimodal image super-resolution model that effectively fuses information from different modalities using coupled sparse priors, leading to superior reconstruction performance.
Contribution
It presents a novel network design inspired by coupled convolutional sparse coding, integrating domain knowledge for improved multimodal super-resolution.
Findings
Outperforms state-of-the-art methods in near-infrared super-resolution
Incorporates domain knowledge through coupled sparse priors
Provides an interpretable deep network architecture
Abstract
Multimodal image super-resolution (SR) is the reconstruction of a high resolution image given a low-resolution observation with the aid of another image modality. While existing deep multimodal models do not incorporate domain knowledge about image SR, we present a multimodal deep network design that integrates coupled sparse priors and allows the effective fusion of information from another modality into the reconstruction process. Our method is inspired by a novel iterative algorithm for coupled convolutional sparse coding, resulting in an interpretable network by design. We apply our model to the super-resolution of near-infrared image guided by RGB images. Experimental results show that our model outperforms state-of-the-art methods.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Image Processing Techniques · Image and Signal Denoising Methods · Advanced Image Fusion Techniques
