Estimating 3D Trajectories from 2D Projections via Disjunctive Factored Four-Way Conditional Restricted Boltzmann Machines

TL;DR

This paper introduces DFFW-CRBM, a novel deep learning model that accurately estimates and predicts 3D trajectories from 2D projections, overcoming challenges like high dimensionality and limited labeled data.

Contribution

The paper presents a new tensor factorization technique integrated into a four-way conditional RBM, improving 3D trajectory estimation from 2D data with reduced computational costs.

Findings

Effective in predicting complex trajectories

Requires limited labeled data

Outperforms existing methods in accuracy

Abstract

Estimation, recognition, and near-future prediction of 3D trajectories based on their two dimensional projections available from one camera source is an exceptionally difficult problem due to uncertainty in the trajectories and environment, high dimensionality of the specific trajectory states, lack of enough labeled data and so on. In this article, we propose a solution to solve this problem based on a novel deep learning model dubbed Disjunctive Factored Four-Way Conditional Restricted Boltzmann Machine (DFFW-CRBM). Our method improves state-of-the-art deep learning techniques for high dimensional time-series modeling by introducing a novel tensor factorization capable of driving forth order Boltzmann machines to considerably lower energy levels, at no computational costs. DFFW-CRBMs are capable of accurately estimating, recognizing, and performing near-future prediction of three-dimensional trajectories from their 2D projections while requiring limited amount of labeled data. We evaluate our method on both simulated and real-world data, showing its effectiveness in predicting and classifying complex ball trajectories and human activities.