Deep Texture and Structure Aware Filtering Network for Image Smoothing

TL;DR

This paper introduces TSAFN, a deep learning-based image smoothing network that effectively distinguishes and preserves structures while removing textures, addressing limitations of previous methods.

Contribution

The paper proposes a novel texture and structure aware filtering network (TSAFN) that combines texture prediction and semantic structure prediction for improved image smoothing.

Findings

TSAFN outperforms existing methods on real images and datasets.

The model effectively separates textures from structures.

Generated dataset enhances training and evaluation.

Abstract

Image smoothing is a fundamental task in computer vision, that aims to retain salient structures and remove insignificant textures. In this paper, we aim to address the fundamental shortcomings of existing image smoothing methods, which cannot properly distinguish textures and structures with similar low-level appearance. While deep learning approaches have started to explore the preservation of structure through image smoothing, existing work does not yet properly address textures. To this end, we generate a large dataset by blending natural textures with clean structure-only images, and then build a texture prediction network (TPN) that predicts the location and magnitude of textures. We then combine the TPN with a semantic structure prediction network (SPN) so that the final texture and structure aware filtering network (TSAFN) is able to identify the textures to remove ("texture-awareness") and the structures to preserve ("structure-awareness"). The proposed model is easy to understand and implement, and shows excellent performance on real images in the wild as well as our generated dataset.