Boundary-aware Decoupled Flow Networks for Realistic Extreme Rescaling

TL;DR

This paper introduces BDFlow, a boundary-aware decoupled flow network that improves image rescaling by modeling high-frequency information as semantic and non-semantic, resulting in more realistic and detailed images.

Contribution

The paper proposes a novel boundary-aware decoupled flow network that separates high-frequency information into semantic and non-semantic parts for improved image rescaling.

Findings

BDFlow outperforms state-of-the-art methods in PSNR and SSIM.

BDFlow achieves higher quality results with lower complexity.

The method effectively captures rich textures and details.

Abstract

Recently developed generative methods, including invertible rescaling network (IRN) based and generative adversarial network (GAN) based methods, have demonstrated exceptional performance in image rescaling. However, IRN-based methods tend to produce over-smoothed results, while GAN-based methods easily generate fake details, which thus hinders their real applications. To address this issue, we propose Boundary-aware Decoupled Flow Networks (BDFlow) to generate realistic and visually pleasing results. Unlike previous methods that model high-frequency information as standard Gaussian distribution directly, our BDFlow first decouples the high-frequency information into \textit{semantic high-frequency} that adheres to a Boundary distribution and \textit{non-semantic high-frequency} counterpart that adheres to a Gaussian distribution. Specifically, to capture semantic high-frequency parts accurately, we use Boundary-aware Mask (BAM) to constrain the model to produce rich textures, while non-semantic high-frequency part is randomly sampled from a Gaussian distribution.Comprehensive experiments demonstrate that our BDFlow significantly outperforms other state-of-the-art methods while maintaining lower complexity. Notably, our BDFlow improves the PSNR by 4.4 dB and the SSIM by 0.1 on average over GRAIN, utilizing only 74% of the parameters and 20% of the computation. The code will be available at https://github.com/THU-Kingmin/BAFlow.