Z-upscaling: Optical Flow Guided Frame Interpolation for Isotropic Reconstruction of 3D EM Volumes

TL;DR

This paper introduces a novel optical flow based method to improve the axial resolution of anisotropic 3D electron microscopy volumes, enabling isotropic reconstruction by interpolating slices using advanced video frame interpolation techniques.

Contribution

It applies optical flow estimation and learning-based video interpolation to 3D EM data, achieving isotropic volumes from anisotropic slices, which is a new approach in electron microscopy reconstruction.

Findings

Successfully enhances axial resolution of EM volumes

Achieves isotropic 3D reconstruction from anisotropic data

Demonstrates effectiveness on publicly available datasets

Abstract

We propose a novel optical flow based approach to enhance the axial resolution of anisotropic 3D EM volumes to achieve isotropic 3D reconstruction. Assuming spatial continuity of 3D biological structures in well aligned EM volumes, we reasoned that optical flow estimation techniques, often applied for temporal resolution enhancement in videos, can be utilized. Pixel level motion is estimated between neighboring 2D slices along z, using spatial gradient flow estimates to interpolate and generate new 2D slices resulting in isotropic voxels. We leverage recent state-of-the-art learning methods for video frame interpolation and transfer learning techniques, and demonstrate the success of our approach on publicly available ultrastructure EM volumes.