Deep Convolution for Irregularly Sampled Temporal Point Clouds

TL;DR

This paper introduces a deep convolutional model for irregularly sampled spatial-temporal data that can predict at arbitrary points, outperforming existing methods in weather and gaming datasets.

Contribution

A novel deep convolutional architecture that directly models irregular space-time point clouds without voxelization, handling multiple entities and flexible query answering.

Findings

Improved prediction accuracy over state-of-the-art baselines.

Demonstrated efficiency in real-world weather data.

Showed flexibility in answering diverse queries.

Abstract

We consider the problem of modeling the dynamics of continuous spatial-temporal processes represented by irregular samples through both space and time. Such processes occur in sensor networks, citizen science, multi-robot systems, and many others. We propose a new deep model that is able to directly learn and predict over this irregularly sampled data, without voxelization, by leveraging a recent convolutional architecture for static point clouds. The model also easily incorporates the notion of multiple entities in the process. In particular, the model can flexibly answer prediction queries about arbitrary space-time points for different entities regardless of the distribution of the training or test-time data. We present experiments on real-world weather station data and battles between large armies in StarCraft II. The results demonstrate the model's flexibility in answering a variety of query types and demonstrate improved performance and efficiency compared to state-of-the-art baselines.