Perception-Distortion Balanced Super-Resolution: A Multi-Objective Optimization Perspective

TL;DR

This paper introduces a multi-objective optimization approach combining evolutionary algorithms and gradient-based methods to balance perceptual quality and distortion in super-resolution, resulting in improved image restoration performance.

Contribution

It proposes a novel optimizer integrating evolutionary algorithms with Adam to better balance conflicting objectives in super-resolution tasks.

Findings

Achieves better perceptual quality than competitors.

Maintains higher reconstruction fidelity.

Generates a set of models with diverse perception-distortion trade-offs.

Abstract

High perceptual quality and low distortion degree are two important goals in image restoration tasks such as super-resolution (SR). Most of the existing SR methods aim to achieve these goals by minimizing the corresponding yet conflicting losses, such as the $\ell_1$ loss and the adversarial loss. Unfortunately, the commonly used gradient-based optimizers, such as Adam, are hard to balance these objectives due to the opposite gradient decent directions of the contradictory losses. In this paper, we formulate the perception-distortion trade-off in SR as a multi-objective optimization problem and develop a new optimizer by integrating the gradient-free evolutionary algorithm (EA) with gradient-based Adam, where EA and Adam focus on the divergence and convergence of the optimization directions respectively. As a result, a population of optimal models with different perception-distortion preferences is obtained. We then design a fusion network to merge these models into a single stronger one for an effective perception-distortion trade-off. Experiments demonstrate that with the same backbone network, the perception-distortion balanced SR model trained by our method can achieve better perceptual quality than its competitors while attaining better reconstruction fidelity. Codes and models can be found at https://github.com/csslc/EA-Adam}{https://github.com/csslc/EA-Adam.