MeVGAN: GAN-based Plugin Model for Video Generation with Applications in Colonoscopy

TL;DR

This paper introduces MeVGAN, a memory-efficient GAN architecture that generates high-quality colonoscopy videos using a plugin approach, aiding medical training and data augmentation.

Contribution

We propose MeVGAN, a novel plugin-based GAN that efficiently generates realistic colonoscopy videos by leveraging a pre-trained 2D GAN and trajectory modeling in noise space.

Findings

MeVGAN produces high-quality synthetic colonoscopy videos.

The model demonstrates memory efficiency suitable for high-resolution video generation.

Generated videos are promising for use in medical training simulators.

Abstract

Video generation is important, especially in medicine, as much data is given in this form. However, video generation of high-resolution data is a very demanding task for generative models, due to the large need for memory. In this paper, we propose Memory Efficient Video GAN (MeVGAN) - a Generative Adversarial Network (GAN) which uses plugin-type architecture. We use a pre-trained 2D-image GAN and only add a simple neural network to construct respective trajectories in the noise space, so that the trajectory forwarded through the GAN model constructs a real-life video. We apply MeVGAN in the task of generating colonoscopy videos. Colonoscopy is an important medical procedure, especially beneficial in screening and managing colorectal cancer. However, because colonoscopy is difficult and time-consuming to learn, colonoscopy simulators are widely used in educating young colonoscopists. We show that MeVGAN can produce good quality synthetic colonoscopy videos, which can be potentially used in virtual simulators.