Multi-modal Image Processing based on Coupled Dictionary Learning

TL;DR

This paper introduces a coupled dictionary learning framework for multi-modal image processing that leverages shared structures across different imaging modalities to enhance tasks like denoising, inpainting, and super-resolution.

Contribution

It proposes a novel multi-modal image processing method based on coupled dictionary learning that captures shared features across modalities in a sparse transform domain.

Findings

Improved image processing performance using multimodal data

Effective capture of shared structures like edges and corners

Enhanced results in denoising, inpainting, and super-resolution tasks

Abstract

In real-world scenarios, many data processing problems often involve heterogeneous images associated with different imaging modalities. Since these multimodal images originate from the same phenomenon, it is realistic to assume that they share common attributes or characteristics. In this paper, we propose a multi-modal image processing framework based on coupled dictionary learning to capture similarities and disparities between different image modalities. In particular, our framework can capture favorable structure similarities across different image modalities such as edges, corners, and other elementary primitives in a learned sparse transform domain, instead of the original pixel domain, that can be used to improve a number of image processing tasks such as denoising, inpainting, or super-resolution. Practical experiments demonstrate that incorporating multimodal information using our framework brings notable benefits.