Enhanced Parking Perception by Multi-Task Fisheye Cross-view Transformers

TL;DR

This paper presents MT F-CVT, a multi-task transformer-based system that enhances parking perception by accurately detecting slots and vehicles using fisheye camera data, suitable for real-time ADAS applications.

Contribution

Introduction of a multi-task transformer framework that processes fisheye surround-view data for detailed parking area perception, combining segmentation and object detection.

Findings

Achieves 20 cm average positional error in real-world scenes.

F-1 score of 0.89 on parking and vehicle detection.

Operates at 16 fps on embedded hardware.

Abstract

Current parking area perception algorithms primarily focus on detecting vacant slots within a limited range, relying on error-prone homographic projection for both labeling and inference. However, recent advancements in Advanced Driver Assistance System (ADAS) require interaction with end-users through comprehensive and intelligent Human-Machine Interfaces (HMIs). These interfaces should present a complete perception of the parking area going from distinguishing vacant slots' entry lines to the orientation of other parked vehicles. This paper introduces Multi-Task Fisheye Cross View Transformers (MT F-CVT), which leverages features from a four-camera fisheye Surround-view Camera System (SVCS) with multihead attentions to create a detailed Bird-Eye View (BEV) grid feature map. Features are processed by both a segmentation decoder and a Polygon-Yolo based object detection decoder for parking slots and vehicles. Trained on data labeled using LiDAR, MT F-CVT positions objects within a 25m x 25m real open-road scenes with an average error of only 20 cm. Our larger model achieves an F-1 score of 0.89. Moreover the smaller model operates at 16 fps on an Nvidia Jetson Orin embedded board, with similar detection results to the larger one. MT F-CVT demonstrates robust generalization capability across different vehicles and camera rig configurations. A demo video from an unseen vehicle and camera rig is available at: https://streamable.com/jjw54x.