FeatMap: Understanding image manipulation in the feature space and its implications for feature space geometry
Elias B. Krey, Nils Neukirch, Nils Strodthoff

TL;DR
This paper investigates the geometric structure of deep neural network feature spaces by applying various input manipulations and assessing the learnability of corresponding feature mappings, revealing linear organization tendencies.
Contribution
It introduces methods to learn and evaluate feature space mappings for diverse image manipulations, highlighting the potential linear structure of feature representations.
Findings
Linear models can effectively approximate complex feature transformations.
Global transformer models outperform local linear models in reconstruction quality.
Feature space mappings reveal a tendency towards linear organization.
Abstract
Intermediate feature representations represent the backbone for the expressivity and adaptability of deep neural networks. However, their geometric structure remains poorly understood. In this submission, we provide indirect insights into this matter by applying a broad selection of manipulations in input space, ranging from geometric and photometric transformations to local masking and semantic manipulations using generative image editing models, and assess the feasibility of learning a mapping in the feature space, mapping from the original to the manipulated feature map. To this end, we devise different types of mappings, from linear to non-linear and local to global mappings and assess both the reconstruction quality of the mapping as well as the semantic content of the mapped representations. We demonstrate the feasibility of learning such mappings for all considered…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
