Legacy Photo Editing with Learned Noise Prior

TL;DR

This paper introduces NEGAN, a noise prior learner for legacy photos, enabling improved denoising, inpainting, and colorization through a novel noise simulation and an integrated editing framework, achieving superior perceptual quality.

Contribution

The paper presents a new learned noise prior model and an integrated image editing framework for restoring legacy photos, using unpaired images and wavelet-based noise matching.

Findings

Achieves state-of-the-art perceptual quality in legacy photo restoration.

Effectively simulates real legacy photo noise using learned noise prior.

Demonstrates superior performance over existing image enhancement methods.

Abstract

There are quite a number of photographs captured under undesirable conditions in the last century. Thus, they are often noisy, regionally incomplete, and grayscale formatted. Conventional approaches mainly focus on one point so that those restoration results are not perceptually sharp or clean enough. To solve these problems, we propose a noise prior learner NEGAN to simulate the noise distribution of real legacy photos using unpaired images. It mainly focuses on matching high-frequency parts of noisy images through discrete wavelet transform (DWT) since they include most of noise statistics. We also create a large legacy photo dataset for learning noise prior. Using learned noise prior, we can easily build valid training pairs by degrading clean images. Then, we propose an IEGAN framework performing image editing including joint denoising, inpainting and colorization based on the estimated noise prior. We evaluate the proposed system and compare it with state-of-the-art image enhancement methods. The experimental results demonstrate that it achieves the best perceptual quality. https://github.com/zhaoyuzhi/Legacy-Photo-Editing-with-Learned-Noise-Prior for the codes and the proposed LP dataset.