SEA: Bridging the Gap Between One- and Two-stage Detector Distillation via SEmantic-aware Alignment

TL;DR

This paper introduces SEA, a semantic-aware distillation framework that effectively bridges one- and two-stage detector training, achieving state-of-the-art results on COCO detection and segmentation tasks.

Contribution

The paper proposes a novel semantic-aware alignment method for detector distillation, addressing pixel imbalance and enhancing semantic bonds between pixels and categories.

Findings

SEA outperforms previous methods on COCO detection benchmarks.

Both RetinaNet and FCOS with SEA surpass their teacher models in AP.

The method generalizes well to instance segmentation tasks.

Abstract

We revisit the one- and two-stage detector distillation tasks and present a simple and efficient semantic-aware framework to fill the gap between them. We address the pixel-level imbalance problem by designing the category anchor to produce a representative pattern for each category and regularize the topological distance between pixels and category anchors to further tighten their semantic bonds. We name our method SEA (SEmantic-aware Alignment) distillation given the nature of abstracting dense fine-grained information by semantic reliance to well facilitate distillation efficacy. SEA is well adapted to either detection pipeline and achieves new state-of-the-art results on the challenging COCO object detection task on both one- and two-stage detectors. Its superior performance on instance segmentation further manifests the generalization ability. Both 2x-distilled RetinaNet and FCOS with ResNet50-FPN outperform their corresponding 3x ResNet101-FPN teacher, arriving 40.64 and 43.06 AP, respectively. Code will be made publicly available.