Real-time Object Detection for Streaming Perception

TL;DR

This paper introduces a real-time streaming perception framework with a novel DualFlow Perception module and trend-aware loss, enabling better future prediction and achieving improved accuracy in autonomous driving scenarios.

Contribution

It proposes a new framework with a DualFlow Perception module and trend-aware loss for enhanced streaming perception in autonomous driving.

Findings

Achieves a 4.9% AP improvement on Argoverse-HD dataset.

Effectively captures moving trends with DualFlow Perception.

Demonstrates the importance of future prediction in real-time perception.

Abstract

Autonomous driving requires the model to perceive the environment and (re)act within a low latency for safety. While past works ignore the inevitable changes in the environment after processing, streaming perception is proposed to jointly evaluate the latency and accuracy into a single metric for video online perception. In this paper, instead of searching trade-offs between accuracy and speed like previous works, we point out that endowing real-time models with the ability to predict the future is the key to dealing with this problem. We build a simple and effective framework for streaming perception. It equips a novel DualFlow Perception module (DFP), which includes dynamic and static flows to capture the moving trend and basic detection feature for streaming prediction. Further, we introduce a Trend-Aware Loss (TAL) combined with a trend factor to generate adaptive weights for objects with different moving speeds. Our simple method achieves competitive performance on Argoverse-HD dataset and improves the AP by 4.9% compared to the strong baseline, validating its effectiveness. Our code will be made available at https://github.com/yancie-yjr/StreamYOLO.