A Fusion of Variational Distribution Priors and Saliency Map Replay for Continual 3D Reconstruction

TL;DR

This paper introduces a continual learning approach for 3D reconstruction from single images, combining variational priors and saliency map replay to improve class retention and resource efficiency.

Contribution

It proposes a novel method integrating Variational Priors with saliency map-based experience replay for continual 3D reconstruction, addressing data and memory constraints.

Findings

Achieves competitive quantitative results

Maintains reasonable reconstruction of previous classes

Efficient memory usage with saliency maps

Abstract

Single-image 3D reconstruction is a research challenge focused on predicting 3D object shapes from single-view images. This task requires significant data acquisition to predict both visible and occluded portions of the shape. Furthermore, learning-based methods face the difficulty of creating a comprehensive training dataset for all possible classes. To this end, we propose a continual learning-based 3D reconstruction method where our goal is to design a model using Variational Priors that can still reconstruct the previously seen classes reasonably even after training on new classes. Variational Priors represent abstract shapes and combat forgetting, whereas saliency maps preserve object attributes with less memory usage. This is vital due to resource constraints in storing extensive training data. Additionally, we introduce saliency map-based experience replay to capture global and distinct object features. Thorough experiments show competitive results compared to established methods, both quantitatively and qualitatively.