Analogical Trajectory Transfer

TL;DR

This paper introduces a method for transferring motion trajectories between 3D scenes by decomposing scenes into clusters, estimating mappings with 3D features, and refining to ensure coherence, enabling applications in AR/VR and robotics.

Contribution

The authors propose a novel, training-free approach that decomposes scenes, predicts hierarchical maps, and refines trajectory transfer, outperforming existing baselines in speed and accuracy.

Findings

Method achieves transfer in approximately 0.6 seconds.

Outperforms LLM, VLM, and scene graph matching baselines.

Enables broad applications in AR/VR and robotics.

Abstract

We study analogical trajectory transfer, where the goal is to translate motion trajectories in one 3D environment to a semantically analogous location in another. Such a capacity would enable machines to perform analogical spatial reasoning, with applications in AR/VR co-presence, content creation, and robotics. However, even semantically similar scenes can still differ substantially in object placement, scale, and layout, so naively matching semantics leads to collisions or geometric distortions. Furthermore, finding where each trajectory point should transfer to has a large search space, as the mapping must preserve semantics and functionality without tearing the trajectory apart or causing collisions. Our key insight is to decompose the problem into spatially segregated subproblems and merge their solutions to produce semantically consistent and spatially coherent transfers. Specifically, we partition scenes into object-centric clusters and estimate cross-scene mappings via hierarchical smooth map prediction, using 3D foundation model features that encode contextual information from object and open-space arrangements. We then combinatorially assemble the per-cluster maps into an initial transfer and refine the result to remove collisions and distortions, yielding a spatially coherent trajectory. Our method does not require training, attains a fast runtime around 0.6 seconds, and outperforms baselines based on LLMs, VLMs, and scene graph matching. We further showcase applications in virtual co-presence, multi-trajectory transfer, camera transfer, and human-to-robot motion transfer, which indicates the broad applicability of our work to AR/VR and robotics.