Learning to Super-Resolve Blurry Images with Events

TL;DR

This paper introduces an event-enhanced super-resolution method for blurry, low-resolution images that leverages event data to produce high-resolution, sharp image sequences, outperforming existing techniques.

Contribution

It proposes a novel event-enhanced degeneration model and a dual sparse learning network, along with an event shuffle-and-merge scheme for sequence super-resolution without extra training.

Findings

Outperforms state-of-the-art methods on synthetic datasets

Effective in real-world scenarios with noisy event data

Generates high-quality, high-resolution image sequences from single blurry inputs

Abstract

Super-Resolution from a single motion Blurred image (SRB) is a severely ill-posed problem due to the joint degradation of motion blurs and low spatial resolution. In this paper, we employ events to alleviate the burden of SRB and propose an Event-enhanced SRB (E-SRB) algorithm, which can generate a sequence of sharp and clear images with High Resolution (HR) from a single blurry image with Low Resolution (LR). To achieve this end, we formulate an event-enhanced degeneration model to consider the low spatial resolution, motion blurs, and event noises simultaneously. We then build an event-enhanced Sparse Learning Network (eSL-Net++) upon a dual sparse learning scheme where both events and intensity frames are modeled with sparse representations. Furthermore, we propose an event shuffle-and-merge scheme to extend the single-frame SRB to the sequence-frame SRB without any additional training process. Experimental results on synthetic and real-world datasets show that the proposed eSL-Net++ outperforms state-of-the-art methods by a large margin. Datasets, codes, and more results are available at https://github.com/ShinyWang33/eSL-Net-Plusplus.