Kandinsky 5.0: A Family of Foundation Models for Image and Video Generation

TL;DR

Kandinsky 5.0 is a comprehensive family of high-resolution image and video generative models, featuring multiple sizes and optimized training techniques for superior quality and speed, with open-source availability.

Contribution

The paper introduces Kandinsky 5.0, a new family of foundation models for image and video generation, including novel training, architectural, and inference optimizations.

Findings

Achieves state-of-the-art performance in image and video synthesis

Demonstrates high generation speed and quality through novel optimizations

Provides open-source code and models for research community

Abstract

This report introduces Kandinsky 5.0, a family of state-of-the-art foundation models for high-resolution image and 10-second video synthesis. The framework comprises three core line-up of models: Kandinsky 5.0 Image Lite - a line-up of 6B parameter image generation models, Kandinsky 5.0 Video Lite - a fast and lightweight 2B parameter text-to-video and image-to-video models, and Kandinsky 5.0 Video Pro - 19B parameter models that achieves superior video generation quality. We provide a comprehensive review of the data curation lifecycle - including collection, processing, filtering and clustering - for the multi-stage training pipeline that involves extensive pre-training and incorporates quality-enhancement techniques such as self-supervised fine-tuning (SFT) and reinforcement learning (RL)-based post-training. We also present novel architectural, training, and inference optimizations that enable Kandinsky 5.0 to achieve high generation speeds and state-of-the-art performance across various tasks, as demonstrated by human evaluation. As a large-scale, publicly available generative framework, Kandinsky 5.0 leverages the full potential of its pre-training and subsequent stages to be adapted for a wide range of generative applications. We hope that this report, together with the release of our open-source code and training checkpoints, will substantially advance the development and accessibility of high-quality generative models for the research community.