Few-shot Single-view 3D Reconstruction with Memory Prior Contrastive Network

TL;DR

This paper introduces MPCN, a novel few-shot 3D reconstruction method that leverages memory prior and contrastive learning to improve performance on unseen categories without requiring category annotations.

Contribution

The paper proposes a Memory Prior Contrastive Network that stores shape priors and uses multi-head attention and contrastive learning to enhance few-shot 3D reconstruction for novel categories.

Findings

Outperforms state-of-the-art methods on synthetic and real datasets.

Effectively handles inter-class variability without category annotations.

Improves retrieval accuracy and feature organization for 3D reconstruction.

Abstract

3D reconstruction of novel categories based on few-shot learning is appealing in real-world applications and attracts increasing research interests. Previous approaches mainly focus on how to design shape prior models for different categories. Their performance on unseen categories is not very competitive. In this paper, we present a Memory Prior Contrastive Network (MPCN) that can store shape prior knowledge in a few-shot learning based 3D reconstruction framework. With the shape memory, a multi-head attention module is proposed to capture different parts of a candidate shape prior and fuse these parts together to guide 3D reconstruction of novel categories. Besides, we introduce a 3D-aware contrastive learning method, which can not only complement the retrieval accuracy of memory network, but also better organize image features for downstream tasks. Compared with previous few-shot 3D reconstruction methods, MPCN can handle the inter-class variability without category annotations. Experimental results on a benchmark synthetic dataset and the Pascal3D+ real-world dataset show that our model outperforms the current state-of-the-art methods significantly.