Hierarchical Mask2Former: Panoptic Segmentation of Crops, Weeds and Leaves

TL;DR

This paper introduces Hierarchical Mask2Former, a novel panoptic segmentation approach for agriculture that accurately identifies crops, weeds, and leaves, enabling targeted interventions and resource-efficient farming.

Contribution

It adapts Mask2Former for agricultural segmentation, achieving high accuracy and developing a more compact, faster architecture with minimal performance loss.

Findings

PQ of 75.99 achieved on agricultural images

Inference speed increased by up to 60% with a compact model

Minimal PQ reduction (<1%) in the compact version

Abstract

Advancements in machine vision that enable detailed inferences to be made from images have the potential to transform many sectors including agriculture. Precision agriculture, where data analysis enables interventions to be precisely targeted, has many possible applications. Precision spraying, for example, can limit the application of herbicide only to weeds, or limit the application of fertiliser only to undernourished crops, instead of spraying the entire field. The approach promises to maximise yields, whilst minimising resource use and harms to the surrounding environment. To this end, we propose a hierarchical panoptic segmentation method to simultaneously identify indicators of plant growth and locate weeds within an image. We adapt Mask2Former, a state-of-the-art architecture for panoptic segmentation, to predict crop, weed and leaf masks. We achieve a PQ{\dag} of 75.99. Additionally, we explore approaches to make the architecture more compact and therefore more suitable for time and compute constrained applications. With our more compact architecture, inference is up to 60% faster and the reduction in PQ{\dag} is less than 1%.