Multi-Resolution Data Fusion for Super Resolution Imaging

TL;DR

This paper introduces a multi-resolution data fusion algorithm for super-resolution imaging that effectively interpolates low-resolution data using unpaired high-resolution data and a novel theoretical framework, improving image quality in materials and biological imaging.

Contribution

The paper presents a new multi-resolution data fusion method utilizing MACE and mismatched back-projectors, with theoretical analysis and practical validation in electron microscopy.

Findings

Reduced artifacts compared to existing methods

Maintains fidelity to measured data

Accurately resolves sub-pixel features

Abstract

Applications in materials and biological imaging are limited by the ability to collect high-resolution data over large areas in practical amounts of time. One solution to this problem is to collect low-resolution data and interpolate to produce a high-resolution image. However, most existing super-resolution algorithms are designed for natural images, often require aligned pairing of high and low-resolution training data, and may not directly incorporate a model of the imaging sensor. In this paper, we present a Multi-resolution Data Fusion (MDF) algorithm for accurate interpolation of low-resolution data at multiple resolutions up to 8x. Our approach uses small quantities of unpaired high-resolution data to train a neural network prior model denoiser and then uses the Multi-Agent Consensus Equilibrium (MACE) problem formulation to balance this denoiser with a forward model agent that promotes fidelity to measured data. A key theoretical novelty is the analysis of mismatched back-projectors, which modify typical forward model updates for computational efficiency or improved image quality. We use MACE to prove that using a mismatched back-projector is equivalent to using a standard back-projector and an appropriately modified prior model. We present electron microscopy results at 4x and 8x interpolation factors that exhibit reduced artifacts relative to existing methods while maintaining fidelity to acquired data and accurately resolving sub-pixel-scale features.