Building a Telescope to Look Into High-Dimensional Image Spaces
Mitch Hill, Erik Nijkamp, Song-Chun Zhu

TL;DR
This paper introduces Attraction-Diffusion, a novel MCMC method inspired by jump-diffusion, to map and visualize the complex local mode structure of high-dimensional image pattern densities learned by neural networks.
Contribution
The work presents a new computational tool, Attraction-Diffusion, for mapping high-dimensional probability densities, revealing the macroscopic structure of image pattern spaces learned by neural models.
Findings
AD efficiently maps highly non-convex densities
Metastable regions contain coherent image groups
Perceptibility influences metastability of image basins
Abstract
An image pattern can be represented by a probability distribution whose density is concentrated on different low-dimensional subspaces in the high-dimensional image space. Such probability densities have an astronomical number of local modes corresponding to typical pattern appearances. Related groups of modes can join to form macroscopic image basins that represent pattern concepts. Recent works use neural networks that capture high-order image statistics to learn Gibbs models capable of synthesizing realistic images of many patterns. However, characterizing a learned probability density to uncover the Hopfield memories of the model, encoded by the structure of the local modes, remains an open challenge. In this work, we present novel computational experiments that map and visualize the local mode structure of Gibbs densities. Efficient mapping requires identifying the global basins…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Neural Networks and Applications · Image and Signal Denoising Methods
