Parsing Images of Overlapping Organisms with Deep Singling-Out Networks

TL;DR

This paper introduces a deep learning method called Singling-Out Networks (SON) for parsing overlapping biological images, effectively estimating individual shapes and disentangling groups without stochastic search or rendering at test time.

Contribution

The paper presents a novel deep feed-forward network that maps local image patches to descriptors sensitive to object shape, enabling efficient parsing of crowded, overlapping organisms.

Findings

Successfully parsed microscopy images of worms, larvae, and bacteria

Handled significant overlaps and crowding in biological images

Avoided stochastic search and rendering during testing

Abstract

This work is motivated by the mostly unsolved task of parsing biological images with multiple overlapping articulated model organisms (such as worms or larvae). We present a general approach that separates the two main challenges associated with such data, individual object shape estimation and object groups disentangling. At the core of the approach is a deep feed-forward singling-out network (SON) that is trained to map each local patch to a vectorial descriptor that is sensitive to the characteristics (e.g. shape) of a central object, while being invariant to the variability of all other surrounding elements. Given a SON, a local image patch can be matched to a gallery of isolated elements using their SON-descriptors, thus producing a hypothesis about the shape of the central element in that patch. The image-level optimization based on integer programming can then pick a subset of the hypotheses to explain (parse) the whole image and disentangle groups of organisms. While sharing many similarities with existing "analysis-by-synthesis" approaches, our method avoids the need for stochastic search in the high-dimensional configuration space and numerous rendering operations at test-time. We show that our approach can parse microscopy images of three popular model organisms (the C.Elegans roundworms, the Drosophila larvae, and the E.Coli bacteria) even under significant crowding and overlaps between organisms. We speculate that the overall approach is applicable to a wider class of image parsing problems concerned with crowded articulated objects, for which rendering training images is possible.