Virtual avatar generation models as world navigators

TL;DR

This paper presents SABR-CLIMB, a diffusion transformer model that generates realistic human movement in climbing environments using a large dataset, aiming to advance virtual avatar applications in robotics, sports, and healthcare.

Contribution

Introduction of SABR-CLIMB, a diffusion transformer model for virtual avatar generation trained on a large proprietary dataset for complex movement tasks.

Findings

Successful simulation of human climbing movements

Effective use of large proprietary dataset NAV-22M

Potential applications in robotics, sports, and healthcare

Abstract

We introduce SABR-CLIMB, a novel video model simulating human movement in rock climbing environments using a virtual avatar. Our diffusion transformer predicts the sample instead of noise in each diffusion step and ingests entire videos to output complete motion sequences. By leveraging a large proprietary dataset, NAV-22M, and substantial computational resources, we showcase a proof of concept for a system to train general-purpose virtual avatars for complex tasks in robotics, sports, and healthcare.