Perceptual-Distortion Balanced Image Super-Resolution is a Multi-Objective Optimization Problem

TL;DR

This paper introduces a multi-objective optimization framework for training super-resolution models that balances perceptual quality and distortion, automating hyperparameter tuning and improving results over existing methods.

Contribution

It proposes MOBOSR, a novel Bayesian optimization-based method that automates loss weight tuning for better perception-distortion trade-offs in super-resolution.

Findings

MOBOSR outperforms state-of-the-art methods in perceptual quality and distortion.

The approach effectively automates hyperparameter tuning, reducing computational costs.

It advances the perception-distortion Pareto frontier in super-resolution tasks.

Abstract

Training Single-Image Super-Resolution (SISR) models using pixel-based regression losses can achieve high distortion metrics scores (e.g., PSNR and SSIM), but often results in blurry images due to insufficient recovery of high-frequency details. Conversely, using GAN or perceptual losses can produce sharp images with high perceptual metric scores (e.g., LPIPS), but may introduce artifacts and incorrect textures. Balancing these two types of losses can help achieve a trade-off between distortion and perception, but the challenge lies in tuning the loss function weights. To address this issue, we propose a novel method that incorporates Multi-Objective Optimization (MOO) into the training process of SISR models to balance perceptual quality and distortion. We conceptualize the relationship between loss weights and image quality assessment (IQA) metrics as black-box objective functions to be optimized within our Multi-Objective Bayesian Optimization Super-Resolution (MOBOSR) framework. This approach automates the hyperparameter tuning process, reduces overall computational cost, and enables the use of numerous loss functions simultaneously. Extensive experiments demonstrate that MOBOSR outperforms state-of-the-art methods in terms of both perceptual quality and distortion, significantly advancing the perception-distortion Pareto frontier. Our work points towards a new direction for future research on balancing perceptual quality and fidelity in nearly all image restoration tasks. The source code and pretrained models are available at: https://github.com/ZhuKeven/MOBOSR.