Texture Reformer: Towards Fast and Universal Interactive Texture Transfer

TL;DR

The Texture Reformer is a fast, universal neural framework for interactive texture transfer that produces high-quality, coherent results with significantly improved speed over existing methods.

Contribution

It introduces a novel multi-stage synthesis process and a learning-free view-specific texture reformation operation for efficient, structure-preserving texture transfer.

Findings

Achieves higher quality texture transfer results.

Operates 2-5 orders of magnitude faster than state-of-the-art methods.

Demonstrates effectiveness across various application scenarios.

Abstract

In this paper, we present the texture reformer, a fast and universal neural-based framework for interactive texture transfer with user-specified guidance. The challenges lie in three aspects: 1) the diversity of tasks, 2) the simplicity of guidance maps, and 3) the execution efficiency. To address these challenges, our key idea is to use a novel feed-forward multi-view and multi-stage synthesis procedure consisting of I) a global view structure alignment stage, II) a local view texture refinement stage, and III) a holistic effect enhancement stage to synthesize high-quality results with coherent structures and fine texture details in a coarse-to-fine fashion. In addition, we also introduce a novel learning-free view-specific texture reformation (VSTR) operation with a new semantic map guidance strategy to achieve more accurate semantic-guided and structure-preserved texture transfer. The experimental results on a variety of application scenarios demonstrate the effectiveness and superiority of our framework. And compared with the state-of-the-art interactive texture transfer algorithms, it not only achieves higher quality results but, more remarkably, also is 2-5 orders of magnitude faster. Code is available at https://github.com/EndyWon/Texture-Reformer.