AVION: Aerial Vision-Language Instruction from Offline Teacher to Prompt-Tuned Network

TL;DR

AVION is a knowledge distillation framework that adapts vision-language models for remote sensing imagery, improving classification and retrieval tasks by leveraging semantic-rich textual prototypes and prompt-tuning.

Contribution

It introduces a novel framework combining semantic-rich textual prototypes with prompt-tuning for effective remote sensing adaptation of vision-language models.

Findings

Improves few-shot classification accuracy.

Enhances cross-modal retrieval mean recall.

Maintains generalization to novel categories.

Abstract

Adapting vision-language models to remote sensing imagery remains challenging due to two key factors: limited semantic coverage in textual representations and insufficient adaptability of visual features. These issues are particularly significant in aerial scenes, which involve various visual appearances and fine-grained object distinctions. We propose AVION, a knowledge distillation framework tailored for remote sensing adaptation of vision-language models. The teacher module constructs semantically rich textual prototypes by collecting descriptions from a large language model and verifying validity using remote sensing image features. The student module integrates lightweight and learnable prompts into both vision and language encoders, guided by the teacher to align embeddings and their cross-modal relationships. Once trained, the student operates independently during inference. Experiments on six optical remote sensing benchmarks show that AVION improves few-shot classification and base-class accuracy without degrading generalization to novel categories. It also enhances mean recall for cross-modal retrieval, with minimal additional trainable parameters.