LIRA: Lifelong Image Restoration from Unknown Blended Distortions

TL;DR

This paper introduces a lifelong image restoration framework that adaptively combines expert models for various distortions, continually learning new tasks without forgetting previous ones, inspired by human neurogenesis.

Contribution

The paper proposes a novel lifelong learning approach for image restoration that integrates expert models and neural growth to handle blended distortions effectively.

Findings

Achieves state-of-the-art results on blended distortion removal.

Maintains old expertise while learning new restoration tasks.

Demonstrates effective continual learning without catastrophic forgetting.

Abstract

Most existing image restoration networks are designed in a disposable way and catastrophically forget previously learned distortions when trained on a new distortion removal task. To alleviate this problem, we raise the novel lifelong image restoration problem for blended distortions. We first design a base fork-join model in which multiple pre-trained expert models specializing in individual distortion removal task work cooperatively and adaptively to handle blended distortions. When the input is degraded by a new distortion, inspired by adult neurogenesis in human memory system, we develop a neural growing strategy where the previously trained model can incorporate a new expert branch and continually accumulate new knowledge without interfering with learned knowledge. Experimental results show that the proposed approach can not only achieve state-of-the-art performance on blended distortions removal tasks in both PSNR/SSIM metrics, but also maintain old expertise while learning new restoration tasks.