Matten: Video Generation with Mamba-Attention

TL;DR

Matten is a novel latent diffusion model with Mamba-Attention architecture that efficiently generates videos by modeling local and global content, achieving competitive performance and scalability.

Contribution

Introduces Matten, a new video generation model combining Mamba-Attention with latent diffusion, offering improved efficiency and scalability over existing Transformer and GAN models.

Findings

Achieves superior FVD scores compared to baseline models.

Demonstrates scalability with increased model complexity.

Maintains competitive performance with minimal computational cost.

Abstract

In this paper, we introduce Matten, a cutting-edge latent diffusion model with Mamba-Attention architecture for video generation. With minimal computational cost, Matten employs spatial-temporal attention for local video content modeling and bidirectional Mamba for global video content modeling. Our comprehensive experimental evaluation demonstrates that Matten has competitive performance with the current Transformer-based and GAN-based models in benchmark performance, achieving superior FVD scores and efficiency. Additionally, we observe a direct positive correlation between the complexity of our designed model and the improvement in video quality, indicating the excellent scalability of Matten.