# Complexer-YOLO: Real-Time 3D Object Detection and Tracking on Semantic Point Clouds

## TL;DR

This paper introduces Complexer-YOLO, a real-time 3D object detection and tracking system that fuses semantic segmentation with detection, improves evaluation speed, and maintains state-of-the-art accuracy on KITTI dataset.

## Contribution

It presents a novel fusion of 3D detection with semantic segmentation, a new evaluation metric (SRTs), and online multi-target tracking for enhanced accuracy and robustness.

## Key findings

- Achieves state-of-the-art results on KITTI dataset.
- Speeds up inference by 20% and halves training time with SRTs.
- First to fuse visual semantic segmentation with 3D object detection.

## Abstract

Accurate detection of 3D objects is a fundamental problem in computer vision and has an enormous impact on autonomous cars, augmented/virtual reality and many applications in robotics. In this work we present a novel fusion of neural network based state-of-the-art 3D detector and visual semantic segmentation in the context of autonomous driving. Additionally, we introduce Scale-Rotation-Translation score (SRTs), a fast and highly parameterizable evaluation metric for comparison of object detections, which speeds up our inference time up to 20\% and halves training time. On top, we apply state-of-the-art online multi target feature tracking on the object measurements to further increase accuracy and robustness utilizing temporal information. Our experiments on KITTI show that we achieve same results as state-of-the-art in all related categories, while maintaining the performance and accuracy trade-off and still run in real-time. Furthermore, our model is the first one that fuses visual semantic with 3D object detection.