Neural Cell Video Synthesis via Optical-Flow Diffusion

TL;DR

This paper introduces a novel method for synthesizing microscopy videos of neuron cells using a diffusion model enhanced with optical flow to improve temporal consistency and quality.

Contribution

It proposes integrating dense optical flow into a diffusion-based video synthesis model to better capture motion dynamics in biomedical microscopy videos.

Findings

Enhanced video quality with optical flow integration

Better temporal consistency in synthesized videos

Guidelines for improving cell video synthesis models

Abstract

The biomedical imaging world is notorious for working with small amounts of data, frustrating state-of-the-art efforts in the computer vision and deep learning worlds. With large datasets, it is easier to make progress we have seen from the natural image distribution. It is the same with microscopy videos of neuron cells moving in a culture. This problem presents several challenges as it can be difficult to grow and maintain the culture for days, and it is expensive to acquire the materials and equipment. In this work, we explore how to alleviate this data scarcity problem by synthesizing the videos. We, therefore, take the recent work of the video diffusion model to synthesize videos of cells from our training dataset. We then analyze the model's strengths and consistent shortcomings to guide us on improving video generation to be as high-quality as possible. To improve on such a task, we propose modifying the denoising function and adding motion information (dense optical flow) so that the model has more context regarding how video frames transition over time and how each pixel changes over time.