Implementation of a perception system for autonomous vehicles using a detection-segmentation network in SoC FPGA

TL;DR

This paper presents an efficient perception system for autonomous vehicles using a detection-segmentation network implemented on an embedded FPGA platform, achieving high accuracy, low power consumption, and compact size for real-time obstacle recognition.

Contribution

It introduces a combined detection and segmentation system on an embedded FPGA platform, optimized for real-time autonomous vehicle perception with high accuracy and low power use.

Findings

Achieves over 97% mAP in object detection

Attains above 90% mIoU in image segmentation

Consumes only 5 watts of power

Abstract

Perception and control systems for autonomous vehicles are an active area of scientific and industrial research. These solutions should be characterised by high efficiency in recognising obstacles and other environmental elements in different road conditions, real-time capability, and energy efficiency. Achieving such functionality requires an appropriate algorithm and a suitable computing platform. In this paper, we have used the MultiTaskV3 detection-segmentation network as the basis for a perception system that can perform both functionalities within a single architecture. It was appropriately trained, quantised, and implemented on the AMD Xilinx Kria KV260 Vision AI embedded platform. By using this device, it was possible to parallelise and accelerate the computations. Furthermore, the whole system consumes relatively little power compared to a CPU-based implementation (an average of 5 watts, compared to the minimum of 55 watts for weaker CPUs, and the small size (119mm x 140mm x 36mm) of the platform allows it to be used in devices where the amount of space available is limited. It also achieves an accuracy higher than 97% of the mAP (mean average precision) for object detection and above 90% of the mIoU (mean intersection over union) for image segmentation. The article also details the design of the Mecanum wheel vehicle, which was used to test the proposed solution in a mock-up city.