Learning without Forgetting for 3D Point Cloud Objects

TL;DR

This paper extends learning without forgetting techniques to 3D point cloud object recognition, addressing challenges unique to 3D data and demonstrating improved knowledge retention across multiple models and datasets.

Contribution

It introduces knowledge distillation with semantic word vectors for 3D data, establishing new baselines for learning without forgetting in 3D point cloud recognition.

Findings

Effective knowledge distillation reduces forgetting in 3D models.

Semantic word vectors improve class retention during training.

New baseline results on multiple datasets and models.

Abstract

When we fine-tune a well-trained deep learning model for a new set of classes, the network learns new concepts but gradually forgets the knowledge of old training. In some real-life applications, we may be interested in learning new classes without forgetting the capability of previous experience. Such learning without forgetting problem is often investigated using 2D image recognition tasks. In this paper, considering the growth of depth camera technology, we address the same problem for the 3D point cloud object data. This problem becomes more challenging in the 3D domain than 2D because of the unavailability of large datasets and powerful pretrained backbone models. We investigate knowledge distillation techniques on 3D data to reduce catastrophic forgetting of the previous training. Moreover, we improve the distillation process by using semantic word vectors of object classes. We observe that exploring the interrelation of old and new knowledge during training helps to learn new concepts without forgetting old ones. Experimenting on three 3D point cloud recognition backbones (PointNet, DGCNN, and PointConv) and synthetic (ModelNet40, ModelNet10) and real scanned (ScanObjectNN) datasets, we establish new baseline results on learning without forgetting for 3D data. This research will instigate many future works in this area.