Metappearance: Meta-Learning for Visual Appearance Reproduction

TL;DR

Metappearance introduces a meta-learning framework that combines the high quality of over-fit methods with the efficiency of generalization, enabling fast and high-quality visual appearance reproduction across various tasks.

Contribution

The paper develops a formalism for applying meta-learning to diverse visual appearance reproduction problems, balancing quality and efficiency.

Findings

Meta-learning improves reproduction quality with reduced runtime.

Framework applies to textures, BRDFs, svBRDFs, and scene lighting.

Provides task-specific guidance for visual appearance methods.

Abstract

There currently exist two main approaches to reproducing visual appearance using Machine Learning (ML): The first is training models that generalize over different instances of a problem, e.g., different images of a dataset. As one-shot approaches, these offer fast inference, but often fall short in quality. The second approach does not train models that generalize across tasks, but rather over-fit a single instance of a problem, e.g., a flash image of a material. These methods offer high quality, but take long to train. We suggest to combine both techniques end-to-end using meta-learning: We over-fit onto a single problem instance in an inner loop, while also learning how to do so efficiently in an outer-loop across many exemplars. To this end, we derive the required formalism that allows applying meta-learning to a wide range of visual appearance reproduction problems: textures, BRDFs, svBRDFs, illumination or the entire light transport of a scene. The effects of meta-learning parameters on several different aspects of visual appearance are analyzed in our framework, and specific guidance for different tasks is provided. Metappearance enables visual quality that is similar to over-fit approaches in only a fraction of their runtime while keeping the adaptivity of general models.