Fine-Grained Image Recognition from Scratch with Teacher-Guided Data Augmentation

TL;DR

This paper demonstrates that high-performance fine-grained image recognition can be achieved from scratch using a novel teacher-guided data augmentation framework, enabling task-specific architectures without reliance on pretrained models.

Contribution

Introduces TGDA, a training framework that combines data-aware augmentation and weak supervision, allowing effective training of FGIR models from scratch and facilitating the development of efficient, task-specific architectures.

Findings

TGDA enables training from scratch to match or surpass pretrained models.

LRNets with TGDA improve accuracy by up to 23% in low-resolution FGIR.

ViTFS-T achieves comparable performance to pretrained ViT B-16 with significantly fewer parameters.

Abstract

Fine-grained image recognition (FGIR) aims to distinguish visually similar sub-categories within a broader class, such as identifying bird species. While most existing FGIR methods rely on backbones pretrained on large-scale datasets like ImageNet, this dependence limits adaptability to resource-constrained environments and hinders the development of task-specific architectures tailored to the unique challenges of FGIR. In this work, we challenge the conventional reliance on pretrained models by demonstrating that high-performance FGIR systems can be trained entirely from scratch. We introduce a novel training framework, TGDA, that integrates data-aware augmentation with weak supervision via a fine-grained-aware teacher model, implemented through knowledge distillation. This framework unlocks the design of task-specific and hardware-aware architectures, including LRNets for low-resolution FGIR and ViTFS, a family of Vision Transformers optimized for efficient inference. Extensive experiments across three FGIR benchmarks over diverse settings involving low-resolution and high-resolution inputs show that our method consistently matches or surpasses state-of-the-art pretrained counterparts. In particular, in the low-resolution setting, LRNets trained with TGDA improve accuracy by up to 23\% over prior methods while requiring up to 20.6x less parameters, lower FLOPs, and significantly less training data. Similarly, ViTFS-T can match the performance of a ViT B-16 pretrained on ImageNet-21k while using 15.3x fewer trainable parameters and requiring orders of magnitudes less data. These results highlight TGDA's potential as an adaptable alternative to pretraining, paving the way for more efficient fine-grained vision systems.