Multiple View Performers for Shape Completion

TL;DR

The paper introduces MVP, a novel Transformer-based architecture for 3D shape completion from sequential views, leveraging memory-efficient attention and achieving registration-free reconstruction.

Contribution

MVP is the first multiple view voxel reconstruction method that does not require registration and employs causal Transformers for 3D shape completion.

Findings

MVP outperforms baseline models in shape completion accuracy.

The model generalizes well across different sequences and scenarios.

It efficiently handles long sequences with size-independent memory.

Abstract

We propose the Multiple View Performer (MVP) - a new architecture for 3D shape completion from a series of temporally sequential views. MVP accomplishes this task by using linear-attention Transformers called Performers. Our model allows the current observation of the scene to attend to the previous ones for more accurate infilling. The history of past observations is compressed via the compact associative memory approximating modern continuous Hopfield memory, but crucially of size independent from the history length. We compare our model with several baselines for shape completion over time, demonstrating the generalization gains that MVP provides. To the best of our knowledge, MVP is the first multiple view voxel reconstruction method that does not require registration of multiple depth views and the first causal Transformer based model for 3D shape completion.