Image Smoothing via Unsupervised Learning

TL;DR

This paper introduces an unsupervised deep learning framework for image smoothing that preserves edges and adapts to different regions, achieving high-quality results efficiently across various applications.

Contribution

It proposes a novel unsupervised learning approach with an energy-based training signal for flexible, high-quality image smoothing without labeled data.

Findings

Achieves comparable or superior results to previous methods.

Runs at 200 fps on modern GPUs.

Effectively handles multiple smoothing applications.

Abstract

Image smoothing represents a fundamental component of many disparate computer vision and graphics applications. In this paper, we present a unified unsupervised (label-free) learning framework that facilitates generating flexible and high-quality smoothing effects by directly learning from data using deep convolutional neural networks (CNNs). The heart of the design is the training signal as a novel energy function that includes an edge-preserving regularizer which helps maintain important yet potentially vulnerable image structures, and a spatially-adaptive Lp flattening criterion which imposes different forms of regularization onto different image regions for better smoothing quality. We implement a diverse set of image smoothing solutions employing the unified framework targeting various applications such as, image abstraction, pencil sketching, detail enhancement, texture removal and content-aware image manipulation, and obtain results comparable with or better than previous methods. Moreover, our method is extremely fast with a modern GPU (e.g, 200 fps for 1280x720 images). Our codes and model are released in https://github.com/fqnchina/ImageSmoothing.