Towards Light Weight Object Detection System

TL;DR

This paper introduces a lightweight transformer-based approach for object detection that reduces latency, improves accuracy through multi-resolution feature fusion, and offers a generalized architecture for future design.

Contribution

It proposes an approximation of self-attention layers, a transformer encoder for feature fusion, and the gFormer abstraction to advance lightweight object detection.

Findings

Reduced latency in transformer-based detection systems

Enhanced accuracy with multi-resolution feature fusion

Provided a flexible architecture for designing new transformers

Abstract

Transformers are a popular choice for classification tasks and as backbones for object detection tasks. However, their high latency brings challenges in their adaptation to lightweight object detection systems. We present an approximation of the self-attention layers used in the transformer architecture. This approximation reduces the latency of the classification system while incurring minimal loss in accuracy. We also present a method that uses a transformer encoder layer for multi-resolution feature fusion. This feature fusion improves the accuracy of the state-of-the-art lightweight object detection system without significantly increasing the number of parameters. Finally, we provide an abstraction for the transformer architecture called Generalized Transformer (gFormer) that can guide the design of novel transformer-like architectures.