TwistNet-2D: Learning Second-Order Channel Interactions via Spiral Twisting for Texture Recognition

TL;DR

TwistNet-2D introduces a lightweight module for texture recognition that captures local pairwise channel interactions with directional displacement, outperforming larger models trained from scratch.

Contribution

It proposes Spiral-Twisted Channel Interaction (STCI), a novel mechanism for encoding cross-position co-occurrence patterns in textures, with minimal parameter increase.

Findings

TwistNet-2D surpasses parameter-matched baselines on texture benchmarks.

It outperforms larger ConvNeXt and Swin Transformer models trained from scratch.

The multi-head structure yields interpretable, orientation-selective features.

Abstract

Second-order feature statistics are central to texture recognition, yet existing mechanisms exhibit a structural tension: bilinear pooling and Gram matrices capture global channel correlations but discard spatial structure, whereas self-attention models capture cross-position relations through weighted sums rather than explicit pairwise products. We propose TwistNet-2D, a lightweight module that computes local pairwise channel products under directional spatial displacement, jointly encoding where features co-occur and how they interact. The core component, Spiral-Twisted Channel Interaction (STCI), shifts one feature map along a prescribed direction before L2-normalized channel multiplication, capturing cross-position co-occurrence patterns that characterize structured and periodic textures. Four directional heads are aggregated through content-adaptive channel reweighting, and the result is injected via a sigmoid-gated residual path with near-zero initialization. TwistNet-2D adds only approximately 3.5% parameters and approximately 2% FLOPs over ResNet-18. To isolate the contribution of architectural inductive bias from that of transfer learning, all models in this study are trained from scratch without ImageNet pretraining. Under this protocol, TwistNet-2D consistently surpasses parameter-matched baselines and substantially larger ConvNeXt and Swin Transformer backbones across four texture and fine-grained recognition benchmarks, while the multi-head structure produces interpretable, orientation-selective representations that align with classical texture analysis.