Spectral Complex Autoencoder Pruning: A Fidelity-Guided Criterion for Extreme Structured Channel Compression

TL;DR

This paper introduces SCAP, a spectral autoencoder-based method for channel pruning in neural networks, effectively reducing FLOPs and parameters while maintaining accuracy by measuring spectral reconstruction fidelity.

Contribution

The paper presents a novel spectral complex autoencoder criterion for structured channel pruning, leveraging frequency domain reconstruction fidelity to identify redundant channels.

Findings

Achieves 90.11% FLOP reduction on VGG16 with minimal accuracy loss.

Uses spectral reconstruction fidelity as an effective importance measure for pruning.

Produces a structurally consistent pruned network with high compression ratios.

Abstract

We propose Spectral Complex Autoencoder Pruning (SCAP), a reconstruction-based criterion that measures functional redundancy at the level of individual output channels. For each convolutional layer, we construct a complex interaction field by pairing the full multi-channel input activation as the real part with a single output-channel activation (spatially aligned and broadcast across input channels) as the imaginary part. We transform this complex field to the frequency domain and train a low-capacity autoencoder to reconstruct normalized spectra. Channels whose spectra are reconstructed with high fidelity are interpreted as lying close to a low-dimensional manifold captured by the autoencoder and are therefore more compressible; conversely, channels with low fidelity are retained as they encode information that cannot be compactly represented by the learned manifold. This yields an importance score (optionally fused with the filter L1 norm) that supports simple threshold-based pruning and produces a structurally consistent pruned network. On VGG16 trained on CIFAR-10, at a fixed threshold of 0.6, we obtain 90.11% FLOP reduction and 96.30% parameter reduction with an absolute Top-1 accuracy drop of 1.67% from a 93.44% baseline after fine-tuning, demonstrating that spectral reconstruction fidelity of complex interaction fields is an effective proxy for channel-level redundancy under aggressive compression.