Stay Positive: Non-Negative Image Synthesis for Augmented Reality

TL;DR

This paper introduces a novel method for non-negative image synthesis tailored for augmented reality, leveraging optical illusions to produce high-quality images under light addition constraints.

Contribution

It proposes a new optimization approach that respects non-negativity constraints and integrates with existing methods for improved image synthesis in AR applications.

Findings

Effective in image-to-image translation tasks

Produces high-quality images with minimal artifacts

Leverages optical illusions for darker regions

Abstract

In applications such as optical see-through and projector augmented reality, producing images amounts to solving non-negative image generation, where one can only add light to an existing image. Most image generation methods, however, are ill-suited to this problem setting, as they make the assumption that one can assign arbitrary color to each pixel. In fact, naive application of existing methods fails even in simple domains such as MNIST digits, since one cannot create darker pixels by adding light. We know, however, that the human visual system can be fooled by optical illusions involving certain spatial configurations of brightness and contrast. Our key insight is that one can leverage this behavior to produce high quality images with negligible artifacts. For example, we can create the illusion of darker patches by brightening surrounding pixels. We propose a novel optimization procedure to produce images that satisfy both semantic and non-negativity constraints. Our approach can incorporate existing state-of-the-art methods, and exhibits strong performance in a variety of tasks including image-to-image translation and style transfer.