ReFu: Recursive Fusion for Exemplar-Free 3D Class-Incremental Learning

TL;DR

ReFu is a new recursive fusion model that combines point clouds and meshes to enable exemplar-free 3D class-incremental learning, improving knowledge retention and integration of multiple data modalities.

Contribution

It introduces a recursive knowledge accumulation method and a fusion module with Pointcloud-guided Mesh Attention, eliminating the need for exemplar storage in 3D continual learning.

Findings

Outperforms existing methods in 3D class-incremental learning tasks.

Effectively integrates point clouds and meshes for robust knowledge retention.

Demonstrates superior stability and accuracy across various datasets.

Abstract

We introduce a novel Recursive Fusion model, dubbed ReFu, designed to integrate point clouds and meshes for exemplar-free 3D Class-Incremental Learning, where the model learns new 3D classes while retaining knowledge of previously learned ones. Unlike existing methods that either rely on storing historical data to mitigate forgetting or focus on single data modalities, ReFu eliminates the need for exemplar storage while utilizing the complementary strengths of both point clouds and meshes. To achieve this, we introduce a recursive method which continuously accumulates knowledge by updating the regularized auto-correlation matrix. Furthermore, we propose a fusion module, featuring a Pointcloud-guided Mesh Attention Layer that learns correlations between the two modalities. This mechanism effectively integrates point cloud and mesh features, leading to more robust and stable continual learning. Experiments across various datasets demonstrate that our proposed framework outperforms existing methods in 3D class-incremental learning.