EdgeCrafter: Compact ViTs for Edge Dense Prediction via Task-Specialized Distillation

TL;DR

EdgeCrafter introduces a compact ViT framework with task-specific distillation, enabling efficient and accurate dense prediction on resource-limited edge devices, outperforming traditional CNNs in several tasks.

Contribution

The paper proposes a unified compact ViT framework with task-specific distillation and edge-aware design, significantly improving edge dense prediction performance.

Findings

ECDet-S achieves 51.7 AP on COCO with fewer than 10M parameters.

ECInsSeg performs comparably to RF-DETR with fewer parameters.

ECPose-X reaches 74.8 AP, surpassing YOLO26Pose-X.

Abstract

Deploying high-performance dense prediction models on resource-constrained edge devices remains challenging due to strict limits on computation and memory. In practice, lightweight systems for object detection, instance segmentation, and pose estimation are still dominated by CNN-based architectures such as YOLO, while compact Vision Transformers (ViTs) often struggle to achieve similarly strong accuracy efficiency tradeoff, even with large scale pretraining. We argue that this gap is largely due to insufficient task specific representation learning in small scale ViTs, rather than an inherent mismatch between ViTs and edge dense prediction. To address this issue, we introduce EdgeCrafter, a unified compact ViT framework for edge dense prediction centered on ECDet, a detection model built from a distilled compact backbone and an edge-friendly encoder decoder design. On the COCO dataset, ECDet-S achieves 51.7 AP with fewer than 10M parameters using only COCO annotations. For instance segmentation, ECInsSeg achieves performance comparable to RF-DETR while using substantially fewer parameters. For pose estimation, ECPose-X reaches 74.8 AP, significantly outperforming YOLO26Pose-X (71.6 AP). These results show that compact ViTs, when paired with task-specialized distillation and edge-aware design, can be a practical and competitive option for edge dense prediction. Code is available at: https://intellindust-ai-lab.github.io/projects/EdgeCrafter/