Quantifying point cloud realism through adversarially learned latent representations

TL;DR

This paper introduces a new adversarially learned metric to quantify the local realism of point clouds, enabling quality assessment and anomaly detection without task-specific annotations.

Contribution

It proposes a novel adversarial approach to learn features for realism scoring of point clouds, applicable to synthetic and real data, without requiring annotations.

Findings

The metric correlates well with perceived realism in experiments.

It enables effective anomaly detection in point cloud data.

The approach demonstrates reliable interpolation between different realism levels.

Abstract

Judging the quality of samples synthesized by generative models can be tedious and time consuming, especially for complex data structures, such as point clouds. This paper presents a novel approach to quantify the realism of local regions in LiDAR point clouds. Relevant features are learned from real-world and synthetic point clouds by training on a proxy classification task. Inspired by fair networks, we use an adversarial technique to discourage the encoding of dataset-specific information. The resulting metric can assign a quality score to samples without requiring any task specific annotations. In a series of experiments, we confirm the soundness of our metric by applying it in controllable task setups and on unseen data. Additional experiments show reliable interpolation capabilities of the metric between data with varying degree of realism. As one important application, we demonstrate how the local realism score can be used for anomaly detection in point clouds.