Contrastive Monotonic Pixel-Level Modulation

TL;DR

MonoPix introduces an unsupervised contrastive framework for pixel-level continuous domain modulation, enabling fine-grained spatial control and fast adaptation in image translation tasks.

Contribution

It proposes a novel contrastive monotonic modulation framework with constraints for pixel-level control, advancing low-level vision and image translation methods.

Findings

Achieves state-of-the-art results on domain translation tasks.

Enables fast domain adaptation with logarithmic complexity.

Provides new solutions for low-light enhancement and noise generation.

Abstract

Continuous one-to-many mapping is a less investigated yet important task in both low-level visions and neural image translation. In this paper, we present a new formulation called MonoPix, an unsupervised and contrastive continuous modulation model, and take a step further to enable a pixel-level spatial control which is critical but can not be properly handled previously. The key feature of this work is to model the monotonicity between controlling signals and the domain discriminator with a novel contrastive modulation framework and corresponding monotonicity constraints. We have also introduced a selective inference strategy with logarithmic approximation complexity and support fast domain adaptations. The state-of-the-art performance is validated on a variety of continuous mapping tasks, including AFHQ cat-dog and Yosemite summer-winter translation. The introduced approach also helps to provide a new solution for many low-level tasks like low-light enhancement and natural noise generation, which is beyond the long-established practice of one-to-one training and inference. Code is available at https://github.com/lukun199/MonoPix.