SinBasis Networks: Matrix-Equivalent Feature Extraction for Wave-Like Optical Spectrograms

TL;DR

SinBasis Networks introduce a unified, physics-inspired framework that enhances feature extraction from wave-like images by reinterpreting convolution and attention as linear transforms, leading to improved robustness and transferability.

Contribution

The paper presents a novel matrix-equivalent approach that integrates sinusoidal priors into CNN, ViT, and Capsule architectures for wave-like image analysis, improving sensitivity and invariance.

Findings

Significant improvements in reconstruction accuracy across diverse datasets.

Enhanced robustness to spatial shifts and cross-domain transfer.

Theoretical analysis confirms increased expressivity and stability.

Abstract

Wave-like images--from attosecond streaking spectrograms to optical spectra, audio mel-spectrograms and periodic video frames--encode critical harmonic structures that elude conventional feature extractors. We propose a unified, matrix-equivalent framework that reinterprets convolution and attention as linear transforms on flattened inputs, revealing filter weights as basis vectors spanning latent feature subspaces. To infuse spectral priors we apply elementwise \(\sin(\cdot)\) mappings to each weight matrix. Embedding these transforms into CNN, ViT and Capsule architectures yields Sin-Basis Networks with heightened sensitivity to periodic motifs and built-in invariance to spatial shifts. Experiments on a diverse collection of wave-like image datasets--including 80,000 synthetic attosecond streaking spectrograms, thousands of Raman, photoluminescence and FTIR spectra, mel-spectrograms from AudioSet and cycle-pattern frames from Kinetics--demonstrate substantial gains in reconstruction accuracy, translational robustness and zero-shot cross-domain transfer. Theoretical analysis via matrix isomorphism and Mercer-kernel truncation quantifies how sinusoidal reparametrization enriches expressivity while preserving stability in data-scarce regimes. Sin-Basis Networks thus offer a lightweight, physics-informed approach to deep learning across all wave-form imaging modalities.