GMT: Guided Mask Transformer for Leaf Instance Segmentation

TL;DR

This paper introduces GMT, a Transformer-based model that incorporates leaf spatial distribution priors to improve leaf instance segmentation accuracy, outperforming existing methods on multiple datasets.

Contribution

The novel Guided Mask Transformer (GMT) effectively integrates spatial priors into segmentation, addressing challenges like occlusion and size variation in leaf images.

Findings

Outperforms state-of-the-art on three plant datasets.

Effectively handles occlusion and size variation.

Leverages spatial priors for improved segmentation.

Abstract

Leaf instance segmentation is a challenging multi-instance segmentation task, aiming to separate and delineate each leaf in an image of a plant. Accurate segmentation of each leaf is crucial for plant-related applications such as the fine-grained monitoring of plant growth and crop yield estimation. This task is challenging because of the high similarity (in shape and colour), great size variation, and heavy occlusions among leaf instances. Furthermore, the typically small size of annotated leaf datasets makes it more difficult to learn the distinctive features needed for precise segmentation. We hypothesise that the key to overcoming the these challenges lies in the specific spatial patterns of leaf distribution. In this paper, we propose the Guided Mask Transformer (GMT), which leverages and integrates leaf spatial distribution priors into a Transformer-based segmentor. These spatial priors are embedded in a set of guide functions that map leaves at different positions into a more separable embedding space. Our GMT consistently outperforms the state-of-the-art on three public plant datasets. Our code is available at https://github.com/vios-s/gmt-leaf-ins-seg.