ASIST: Annotation-free synthetic instance segmentation and tracking for microscope video analysis

TL;DR

This paper introduces ASIST, a novel annotation-free deep learning framework for instance segmentation and tracking of microvilli in microscope videos, reducing manual labeling efforts and outperforming supervised methods.

Contribution

It presents the first study on microvilli segmentation and tracking using embedding-based deep learning with an annotation-free synthetic learning approach.

Findings

Achieved superior performance over supervised methods

Proposed a new annotation-free video analysis paradigm

Integrated embedding-based segmentation with synthetic learning

Abstract

Instance object segmentation and tracking provide comprehensive quantification of objects across microscope videos. The recent single-stage pixel-embedding based deep learning approach has shown its superior performance compared with "segment-then-associate" two-stage solutions. However, one major limitation of applying a supervised pixel-embedding based method to microscope videos is the resource-intensive manual labeling, which involves tracing hundreds of overlapped objects with their temporal associations across video frames. Inspired by the recent generative adversarial network (GAN) based annotation-free image segmentation, we propose a novel annotation-free synthetic instance segmentation and tracking (ASIST) algorithm for analyzing microscope videos of sub-cellular microvilli. The contributions of this paper are three-fold: (1) proposing a new annotation-free video analysis paradigm is proposed. (2) aggregating the embedding based instance segmentation and tracking with annotation-free synthetic learning as a holistic framework; and (3) to the best of our knowledge, this is first study to investigate microvilli instance segmentation and tracking using embedding based deep learning. From the experimental results, the proposed annotation-free method achieved superior performance compared with supervised learning.