HybridNets: End-to-End Perception Network

TL;DR

HybridNets is an end-to-end perception network designed for autonomous driving that efficiently performs multiple tasks like object detection, lane detection, and drivable area segmentation with improved accuracy and real-time capability.

Contribution

The paper introduces novel optimizations including a weighted bidirectional feature network, customized anchors, and an efficient training strategy for multi-task perception in autonomous driving.

Findings

Achieves 77.3 mAP on Berkeley DeepDrive dataset.

Outperforms prior methods in lane detection with 31.6 mIoU.

Operates in real-time with 12.83 million parameters.

Abstract

End-to-end Network has become increasingly important in multi-tasking. One prominent example of this is the growing significance of a driving perception system in autonomous driving. This paper systematically studies an end-to-end perception network for multi-tasking and proposes several key optimizations to improve accuracy. First, the paper proposes efficient segmentation head and box/class prediction networks based on weighted bidirectional feature network. Second, the paper proposes automatically customized anchor for each level in the weighted bidirectional feature network. Third, the paper proposes an efficient training loss function and training strategy to balance and optimize network. Based on these optimizations, we have developed an end-to-end perception network to perform multi-tasking, including traffic object detection, drivable area segmentation and lane detection simultaneously, called HybridNets, which achieves better accuracy than prior art. In particular, HybridNets achieves 77.3 mean Average Precision on Berkeley DeepDrive Dataset, outperforms lane detection with 31.6 mean Intersection Over Union with 12.83 million parameters and 15.6 billion floating-point operations. In addition, it can perform visual perception tasks in real-time and thus is a practical and accurate solution to the multi-tasking problem. Code is available at https://github.com/datvuthanh/HybridNets.