Learning Fourier shapes to probe the geometric world of deep neural networks

TL;DR

This paper introduces a Fourier-based framework to generate and analyze geometric shapes in neural networks, revealing their interpretability, semantic power, and vulnerability to adversarial attacks.

Contribution

It presents a novel end-to-end differentiable method using Fourier series to optimize and interpret shapes in DNNs, advancing geometric understanding and adversarial testing.

Findings

Optimized shapes serve as high-confidence semantic classifiers.

Shapes act as precise interpretability tools for salient regions.

The framework enables new adversarial attacks on visual tasks.

Abstract

While both shape and texture are fundamental to visual recognition, research on deep neural networks (DNNs) has predominantly focused on the latter, leaving their geometric understanding poorly probed. Here, we show: first, that optimized shapes can act as potent semantic carriers, generating high-confidence classifications from inputs defined purely by their geometry; second, that they are high-fidelity interpretability tools that precisely isolate a model's salient regions; and third, that they constitute a new, generalizable adversarial paradigm capable of deceiving downstream visual tasks. This is achieved through an end-to-end differentiable framework that unifies a powerful Fourier series to parameterize arbitrary shapes, a winding number-based mapping to translate them into the pixel grid required by DNNs, and signal energy constraints that enhance optimization efficiency while ensuring physically plausible shapes. Our work provides a versatile framework for probing the geometric world of DNNs and opens new frontiers for challenging and understanding machine perception.