Beyond Linear Bottlenecks: Spline-Based Knowledge Distillation for Culturally Diverse Art Style Classification

TL;DR

This paper introduces a spline-based knowledge distillation method using Kolmogorov-Arnold Networks to better model complex, nonlinear art style features, significantly improving classification accuracy in computational aesthetics.

Contribution

It replaces traditional linear projection heads with KANs in dual-teacher frameworks, enabling better modeling of nonlinear style-feature interactions for art style classification.

Findings

Outperforms baseline dual-teacher models in accuracy

KANs effectively model nonlinear style manifolds

Improves linear probe accuracy over MLP projections

Abstract

Art style classification remains a formidable challenge in computational aesthetics due to the scarcity of expertly labeled datasets and the intricate, often nonlinear interplay of stylistic elements. While recent dual-teacher self-supervised frameworks reduce reliance on labeled data, their linear projection layers and localized focus struggle to model global compositional context and complex style-feature interactions. We enhance the dual-teacher knowledge distillation framework to address these limitations by replacing conventional MLP projection and prediction heads with Kolmogorov-Arnold Networks (KANs). Our approach retains complementary guidance from two teacher networks, one emphasizing localized texture and brushstroke patterns, the other capturing broader stylistic hierarchies while leveraging KANs' spline-based activations to model nonlinear feature correlations with mathematical precision. Experiments on WikiArt and Pandora18k demonstrate that our approach outperforms the base dual teacher architecture in Top-1 accuracy. Our findings highlight the importance of KANs in disentangling complex style manifolds, leading to better linear probe accuracy than MLP projections.