An automated workflow for parallel processing of large multiview SPIM recordings

TL;DR

This paper presents an automated, scalable workflow for processing large multiview LSFM datasets efficiently on both single workstations and high performance clusters, significantly reducing processing time.

Contribution

It introduces a flexible, dependency-resolving pipeline using snakemake for automated parallel processing of complex LSFM data.

Findings

Enables processing of large multiview LSFM data in a fraction of the original time.

Supports both workstation and HPC environments for flexible deployment.

Automates multiple processing steps with dependency management.

Abstract

Multiview light sheet fluorescence microscopy (LSFM) allows to image developing organisms in 3D at unprecedented temporal resolution over long periods of time. The resulting massive amounts of raw image data requires extensive processing interactively via dedicated graphical user interface (GUI) applications. The consecutive processing steps can be easily automated and the individual time points can be processed independently, which lends itself to trivial parallelization on a high performance cluster (HPC). Here we introduce an automated workflow for processing large multiview, multi-channel, multi-illumination time-lapse LSFM data on a single workstation or in parallel on a HPC. The pipeline relies on snakemake to resolve dependencies among consecutive processing steps and can be easily adapted to any cluster environment for processing LSFM data in a fraction of the time required to collect it.