Bypass Enhancement RGB Stream Model for Pedestrian Action Recognition of Autonomous Vehicles

TL;DR

This paper introduces a bypass enhanced RGB stream model for pedestrian action recognition in autonomous vehicles, focusing on improving real-time performance and reducing computational complexity without sacrificing accuracy.

Contribution

The study proposes a novel two-branch RGB and optical flow feature extraction model with bypass enhancement and distillation, optimizing real-time pedestrian action recognition.

Findings

Significantly improved real-time performance

Maintained high accuracy in pedestrian action recognition

Validated effectiveness through experiments

Abstract

Pedestrian action recognition and intention prediction is one of the core issues in the field of autonomous driving. In this research field, action recognition is one of the key technologies. A large number of scholars have done a lot of work to im-prove the accuracy of the algorithm for the task. However, there are relatively few studies and improvements in the computational complexity of algorithms and sys-tem real-time. In the autonomous driving application scenario, the real-time per-formance and ultra-low latency of the algorithm are extremely important evalua-tion indicators, which are directly related to the availability and safety of the au-tonomous driving system. To this end, we construct a bypass enhanced RGB flow model, which combines the previous two-branch algorithm to extract RGB feature information and optical flow feature information respectively. In the train-ing phase, the two branches are merged by distillation method, and the bypass enhancement is combined in the inference phase to ensure accuracy. The real-time behavior of the behavior recognition algorithm is significantly improved on the premise that the accuracy does not decrease. Experiments confirm the superiority and effectiveness of our algorithm.