ASIST: Automatic Semantically Invariant Scene Transformation

TL;DR

ASIST is a novel method that transforms point clouds by replacing objects with semantically equivalent ones, useful in VR, scan repair, and robotics, achieved through a unified optimization approach.

Contribution

It introduces a unified formulation for semantic labeling and object replacement in point clouds, with efficient numerical solutions and validation on synthetic and real datasets.

Findings

Effective object replacement demonstrated on diverse datasets.

Outperforms recent methods in semantic consistency and accuracy.

Applicable to real-world and synthetic point cloud data.

Abstract

We present ASIST, a technique for transforming point clouds by replacing objects with their semantically equivalent counterparts. Transformations of this kind have applications in virtual reality, repair of fused scans, and robotics. ASIST is based on a unified formulation of semantic labeling and object replacement; both result from minimizing a single objective. We present numerical tools for the efficient solution of this optimization problem. The method is experimentally assessed on new datasets of both synthetic and real point clouds, and is additionally compared to two recent works on object replacement on data from the corresponding papers.