Initialization and Alignment for Adversarial Texture Optimization

TL;DR

This paper enhances adversarial texture optimization by introducing explicit initialization and alignment procedures, significantly improving robustness and texture quality on challenging real-world data.

Contribution

It proposes a novel initialization and alignment method to improve adversarial texture optimization robustness against complex geometry and misaligned data.

Findings

Achieved 7.8% improvement in perceptual quality

Achieved 11.1% improvement in image sharpness

Validated on 11 scenes with 2807 frames

Abstract

While recovery of geometry from image and video data has received a lot of attention in computer vision, methods to capture the texture for a given geometry are less mature. Specifically, classical methods for texture generation often assume clean geometry and reasonably well-aligned image data. While very recent methods, e.g., adversarial texture optimization, better handle lower-quality data obtained from hand-held devices, we find them to still struggle frequently. To improve robustness, particularly of recent adversarial texture optimization, we develop an explicit initialization and an alignment procedure. It deals with complex geometry due to a robust mapping of the geometry to the texture map and a hard-assignment-based initialization. It deals with misalignment of geometry and images by integrating fast image-alignment into the texture refinement optimization. We demonstrate efficacy of our texture generation on a dataset of 11 scenes with a total of 2807 frames, observing 7.8% and 11.1% relative improvements regarding perceptual and sharpness measurements.