Frequency-Adaptive Discrete Cosine-ViT-ResNet Architecture for Sparse-Data Vision

TL;DR

This paper introduces a hybrid deep-learning framework with adaptive frequency-domain DCT filtering, ViT and ResNet backbones, and Bayesian classification to improve rare animal image classification with scarce data.

Contribution

It presents the first adaptive frequency selection mechanism in DCT preprocessing tailored for deep backbones in sparse-data vision tasks.

Findings

Outperforms conventional CNN and fixed-band DCT pipelines.

Achieves state-of-the-art accuracy on a 50-class wildlife dataset.

Effectively handles extreme sample scarcity.

Abstract

A major challenge in rare animal image classification is the scarcity of data, as many species usually have only a small number of labeled samples. To address this challenge, we designed a hybrid deep-learning framework comprising a novel adaptive DCT preprocessing module, ViT-B16 and ResNet50 backbones, and a Bayesian linear classification head. To our knowledge, we are the first to introduce an adaptive frequency-domain selection mechanism that learns optimal low-, mid-, and high-frequency boundaries suited to the subsequent backbones. Our network first captures image frequency-domain cues via this adaptive DCT partitioning. The adaptively filtered frequency features are then fed into ViT-B16 to model global contextual relationships, while ResNet50 concurrently extracts local, multi-scale spatial representations from the original image. A cross-level fusion strategy seamlessly integrates these frequency- and spatial-domain embeddings, and the fused features are passed through a Bayesian linear classifier to output the final category predictions. On our self-built 50-class wildlife dataset, this approach outperforms conventional CNN and fixed-band DCT pipelines, achieving state-of-the-art accuracy under extreme sample scarcity.