Knowledge Distillation for Object Detection: from generic to remote sensing datasets

TL;DR

This paper evaluates various knowledge distillation methods for object detection in remote sensing datasets, highlighting their performance variations and emphasizing the importance of result aggregation and validation.

Contribution

It systematically compares off-the-shelf distillation techniques on remote sensing datasets, extending their evaluation beyond generic computer vision benchmarks.

Findings

High performance variation among methods

Result aggregation improves reliability

Cross-validation is crucial for remote sensing datasets

Abstract

Knowledge distillation, a well-known model compression technique, is an active research area in both computer vision and remote sensing communities. In this paper, we evaluate in a remote sensing context various off-the-shelf object detection knowledge distillation methods which have been originally developed on generic computer vision datasets such as Pascal VOC. In particular, methods covering both logit mimicking and feature imitation approaches are applied for vehicle detection using the well-known benchmarks such as xView and VEDAI datasets. Extensive experiments are performed to compare the relative performance and interrelationships of the methods. Experimental results show high variations and confirm the importance of result aggregation and cross validation on remote sensing datasets.