AI Powered High Quality Text to Video Generation with Enhanced Temporal Consistency

TL;DR

This paper introduces MOVAI, a hierarchical framework for text-to-video generation that significantly improves temporal consistency, visual quality, and semantic control by integrating scene understanding and diffusion models.

Contribution

The paper presents a novel hierarchical framework with a scene parser, attention mechanism, and refinement module, advancing the state-of-the-art in text-to-video synthesis.

Findings

Achieves 15.3% improvement in LPIPS quality metric.

Improves FVD by 12.7%, indicating better temporal coherence.

Outperforms existing methods in user preference studies by 18.9%.

Abstract

Text to video generation has emerged as a critical frontier in generative artificial intelligence, yet existing approaches struggle with maintaining temporal consistency, compositional understanding, and fine grained control over visual narratives. We present MOVAI (Multimodal Original Video AI), a novel hierarchical framework that integrates compositional scene understanding with temporal aware diffusion models for high fidelity text to video synthesis. Our approach introduces three key innovations: (1) a Compositional Scene Parser (CSP) that decomposes textual descriptions into hierarchical scene graphs with temporal annotations, (2) a Temporal-Spatial Attention Mechanism (TSAM) that ensures coherent motion dynamics across frames while preserving spatial details, and (3) a Progressive Video Refinement (PVR) module that iteratively enhances video quality through multi-scale temporal reasoning. Extensive experiments on standard benchmarks demonstrate that MOVAI achieves state-of-the-art performance, improving video quality metrics by 15.3% in LPIPS, 12.7% in FVD, and 18.9% in user preference studies compared to existing methods. Our framework shows particular strength in generating complex multi-object scenes with realistic temporal dynamics and fine-grained semantic control.