PAND: Prompt-Aware Neighborhood Distillation for Lightweight Fine-Grained Visual Classification

TL;DR

PAND is a novel two-stage framework that improves lightweight models for fine-grained visual classification by decoupling semantic calibration from structural transfer, leading to superior performance on multiple benchmarks.

Contribution

It introduces Prompt-Aware Neighborhood Distillation, a method that enhances knowledge transfer from large VLMs to lightweight networks through semantic and structural alignment.

Findings

Achieves 76.09% accuracy on CUB-200 with ResNet-18.

Outperforms state-of-the-art methods on four FGVC benchmarks.

Demonstrates significant improvement over baseline VL2Lite.

Abstract

Distilling knowledge from large Vision-Language Models (VLMs) into lightweight networks is crucial yet challenging in Fine-Grained Visual Classification (FGVC), due to the reliance on fixed prompts and global alignment. To address this, we propose PAND (Prompt-Aware Neighborhood Distillation), a two-stage framework that decouples semantic calibration from structural transfer. First, we incorporate Prompt-Aware Semantic Calibration to generate adaptive semantic anchors. Second, we introduce a neighborhood-aware structural distillation strategy to constrain the student's local decision structure. PAND consistently outperforms state-of-the-art methods on four FGVC benchmarks. Notably, our ResNet-18 student achieves 76.09% accuracy on CUB-200, surpassing the strong baseline VL2Lite by 3.4%. Code is available at https://github.com/LLLVTA/PAND.