Can No-Reference Quality-Assessment Methods Serve as Perceptual Losses for Super-Resolution?

TL;DR

This paper explores using no-reference image quality assessment methods as perceptual losses in super-resolution, showing potential benefits but also challenges like artifacts, which can be mitigated with special training procedures.

Contribution

It is the first to investigate the use of no-reference quality assessment methods as perceptual losses in super-resolution models.

Findings

Direct optimization causes artifacts in super-resolution images.

A special training procedure reduces artifacts.

No-reference methods can serve as perceptual losses with proper training.

Abstract

Perceptual losses play an important role in constructing deep-neural-network-based methods by increasing the naturalness and realism of processed images and videos. Use of perceptual losses is often limited to LPIPS, a fullreference method. Even though deep no-reference image-qualityassessment methods are excellent at predicting human judgment, little research has examined their incorporation in loss functions. This paper investigates direct optimization of several video-superresolution models using no-reference image-quality-assessment methods as perceptual losses. Our experimental results show that straightforward optimization of these methods produce artifacts, but a special training procedure can mitigate them.