Pik-Fix: Restoring and Colorizing Old Photos

TL;DR

Pik-Fix is a novel deep learning framework that restores and colorizes old photos by combining restoration, color transfer, and colorization modules, leveraging a new dataset of paired degraded and pristine images.

Contribution

It introduces a reference-based end-to-end learning system for restoring and colorizing old photos, utilizing a new dataset and color histogram priors to reduce data requirements.

Findings

Outperforms previous state-of-the-art methods in qualitative and quantitative evaluations.

Effectively restores severely degraded old photos with colorization.

Demonstrates the benefit of using reference images and color histogram priors.

Abstract

Restoring and inpainting the visual memories that are present, but often impaired, in old photos remains an intriguing but unsolved research topic. Decades-old photos often suffer from severe and commingled degradation such as cracks, defocus, and color-fading, which are difficult to treat individually and harder to repair when they interact. Deep learning presents a plausible avenue, but the lack of large-scale datasets of old photos makes addressing this restoration task very challenging. Here we present a novel reference-based end-to-end learning framework that is able to both repair and colorize old, degraded pictures. Our proposed framework consists of three modules: a restoration sub-network that conducts restoration from degradations, a similarity network that performs color histogram matching and color transfer, and a colorization subnet that learns to predict the chroma elements of images conditioned on chromatic reference signals. The overall system makes uses of color histogram priors from reference images, which greatly reduces the need for large-scale training data. We have also created a first-of-a-kind public dataset of real old photos that are paired with ground truth ''pristine'' photos that have been manually restored by PhotoShop experts. We conducted extensive experiments on this dataset and synthetic datasets, and found that our method significantly outperforms previous state-of-the-art models using both qualitative comparisons and quantitative measurements. The code is available at https://github.com/DerrickXuNu/Pik-Fix.