RFNet-4D++: Joint Object Reconstruction and Flow Estimation from 4D Point Clouds with Cross-Attention Spatio-Temporal Features

TL;DR

RFNet-4D++ is a novel neural network architecture that jointly reconstructs 3D objects and estimates their motion flows from 4D point cloud sequences, leveraging cross-attention spatio-temporal features for improved accuracy and efficiency.

Contribution

The paper introduces RFNet-4D++, a new joint learning framework for object reconstruction and flow estimation from 4D point clouds, utilizing cross-attention mechanisms and an unsupervised flow learning module.

Findings

Achieves state-of-the-art results in object reconstruction and flow estimation.

Runs significantly faster than existing methods during training and inference.

Validated on benchmark datasets with extensive experiments.

Abstract

Object reconstruction from 3D point clouds has been a long-standing research problem in computer vision and computer graphics, and achieved impressive progress. However, reconstruction from time-varying point clouds (a.k.a. 4D point clouds) is generally overlooked. In this paper, we propose a new network architecture, namely RFNet-4D++, that jointly reconstructs objects and their motion flows from 4D point clouds. The key insight is simultaneously performing both tasks via learning of spatial and temporal features from a sequence of point clouds can leverage individual tasks, leading to improved overall performance. To prove this ability, we design a temporal vector field learning module using an unsupervised learning approach for flow estimation task, leveraged by supervised learning of spatial structures for object reconstruction. Extensive experiments and analyses on benchmark datasets validated the effectiveness and efficiency of our method. As shown in experimental results, our method achieves state-of-the-art performance on both flow estimation and object reconstruction while performing much faster than existing methods in both training and inference. Our code and data are available at https://github.com/hkust-vgd/RFNet-4D