DeepTracking-Net: 3D Tracking with Unsupervised Learning of Continuous Flow

TL;DR

DeepTracking-Net introduces an unsupervised deep learning framework for 3D shape tracking that captures spatio-temporal correspondences using a novel temporal-aware descriptor, outperforming existing methods on simulated and real datasets.

Contribution

The paper presents a novel unsupervised 3D tracking method using deep neural networks and a new temporal-aware descriptor, enabling continuous flow estimation in time-varying 3D shapes.

Findings

Outperforms current supervised state-of-the-art methods.

Effectively captures spatio-temporal correspondences in 3D data.

Introduces a new synthetic dataset SynMotions for 3D tracking research.

Abstract

This paper deals with the problem of 3D tracking, i.e., to find dense correspondences in a sequence of time-varying 3D shapes. Despite deep learning approaches have achieved promising performance for pairwise dense 3D shapes matching, it is a great challenge to generalize those approaches for the tracking of 3D time-varying geometries. In this paper, we aim at handling the problem of 3D tracking, which provides the tracking of the consecutive frames of 3D shapes. We propose a novel unsupervised 3D shape registration framework named DeepTracking-Net, which uses the deep neural networks (DNNs) as auxiliary functions to produce spatially and temporally continuous displacement fields for 3D tracking of objects in a temporal order. Our key novelty is that we present a novel temporal-aware correspondence descriptor (TCD) that captures spatio-temporal essence from consecutive 3D point cloud frames. Specifically, our DeepTracking-Net starts with optimizing a randomly initialized latent TCD. The TCD is then decoded to regress a continuous flow (i.e. a displacement vector field) which assigns a motion vector to every point of time-varying 3D shapes. Our DeepTracking-Net jointly optimizes TCDs and DNNs' weights towards the minimization of an unsupervised alignment loss. Experiments on both simulated and real data sets demonstrate that our unsupervised DeepTracking-Net outperforms the current supervised state-of-the-art method. In addition, we prepare a new synthetic 3D data, named SynMotions, to the 3D tracking and recognition community.