MetaDecorator: Generating Immersive Virtual Tours through Multimodality

TL;DR

MetaDecorator is a framework that uses text prompts and image synthesis to personalize and enhance virtual tours from 360-degree panoramas, integrating LLMs and haptics for immersive VR experiences.

Contribution

It introduces a novel method for transforming static panoramas into styled, engaging virtual environments using multimodal AI techniques.

Findings

Enhanced realism and engagement in virtual tours

Effective personalization through text-driven prompts

Integration of LLMs and haptics improves immersion

Abstract

MetaDecorator, is a framework that empowers users to personalize virtual spaces. By leveraging text-driven prompts and image synthesis techniques, MetaDecorator adorns static panoramas captured by 360{\deg} imaging devices, transforming them into uniquely styled and visually appealing environments. This significantly enhances the realism and engagement of virtual tours compared to traditional offerings. Beyond the core framework, we also discuss the integration of Large Language Models (LLMs) and haptics in the VR application to provide a more immersive experience.