Efficient Online Transfer Learning for 3D Object Classification in Autonomous Driving

TL;DR

This paper introduces a multi-modal online learning system that transfers visual detection knowledge to LiDAR-based 3D object classification, enabling real-time, high-performance model updates in autonomous driving environments.

Contribution

It presents a novel multi-modal online transfer learning approach utilizing Online Random Forests for improved 3D object classification from LiDAR data.

Findings

System achieves high accuracy in real-time 3D object classification

Effective transfer of visual detection capabilities to LiDAR data

Suitable for in-situ deployment in autonomous vehicles

Abstract

Autonomous driving has achieved rapid development over the last few decades, including the machine perception as an important issue of it. Although object detection based on conventional cameras has achieved remarkable results in 2D/3D, non-visual sensors such as 3D LiDAR still have incomparable advantages in the accuracy of object position detection. However, the challenge also exists with the difficulty in properly interpreting point cloud generated by LiDAR. This paper presents a multi-modal-based online learning system for 3D LiDAR-based object classification in urban environments, including cars, cyclists and pedestrians. The proposed system aims to effectively transfer the mature detection capabilities based on visual sensors to the new model learning based on non-visual sensors through a multi-target tracker (i.e. using one sensor to train another). In particular, it integrates the Online Random Forests (ORF) [1] method, which inherently has the abilities of fast and multi-class learning. Through experiments, we show that our system is capable of learning a high-performance model for LiDAR-based 3D object classification on-the-fly, which is especially suitable for robotics in-situ deployment while responding to the widespread challenge of insufficient detector generalization capabilities.