POSE: Pose estimation Of virtual Sync Exhibit system

TL;DR

This paper presents POSE, a portable MetaVerse system that uses AI-driven 3D pose estimation to enable virtual avatars to perform synchronized actions and interact with the environment, reducing reliance on traditional controllers.

Contribution

The work introduces a novel platform that leverages 3D pose estimation for controlling virtual avatars, offering a modular design with reduced latency for interactive MetaVerse experiences.

Findings

Effective real-time pose estimation for avatar control

Reduced latency through multi-process architecture

Enhanced user interaction without traditional controllers

Abstract

This work is a portable MetaVerse implementation, and we use 3D pose estimation with AI to make virtual avatars do synchronized actions and interact with the environment. The motivation is that we find it inconvenient to use joysticks and sensors when playing with fitness rings. In order to replace joysticks and reduce costs, we developed a platform that can control virtual avatars through pose estimation to identify the movements of real people, and we also implemented a multi-process to achieve modularization and reduce the overall latency.