MIRACLE3D: Memory-efficient Integrated Robust Approach for Continual Learning on Point Clouds via Shape Model Construction

TL;DR

MIRACLE3D introduces a memory-efficient, privacy-preserving continual learning framework for 3D object classification that constructs compact shape models and employs Gradient Mode Regularization, achieving state-of-the-art results with minimal memory use.

Contribution

The paper presents a novel shape model construction method for continual learning in 3D, reducing memory needs and enhancing privacy while improving robustness.

Findings

Achieves state-of-the-art accuracy on ModelNet40, ShapeNet, and ScanNet datasets.

Uses only 15% of the memory of competing methods on ModelNet40 and ShapeNet.

Maintains high performance with just 8.5% memory on ScanNet.

Abstract

In this paper, we introduce a novel framework for memory-efficient and privacy-preserving continual learning in 3D object classification. Unlike conventional memory-based approaches in continual learning that require storing numerous exemplars, our method constructs a compact shape model for each class, retaining only the mean shape along with a few key modes of variation. This strategy not only enables the generation of diverse training samples while drastically reducing memory usage but also enhances privacy by eliminating the need to store original data. To further improve model robustness against input variations, an issue common in 3D domains due to the absence of strong backbones and limited training data, we incorporate Gradient Mode Regularization. This technique enhances model stability and broadens classification margins, resulting in accuracy improvements. We validate our approach through extensive experiments on the ModelNet40, ShapeNet, and ScanNet datasets, where we achieve state-of-the-art performance. Notably, our method consumes only 15% of the memory required by competing methods on the ModelNet40 and ShapeNet, while achieving comparable performance on the challenging ScanNet dataset with just 8.5% of the memory. These results underscore the scalability, effectiveness, and privacy-preserving strengths of our framework for 3D object classification.