Leaf Angle Estimation using Mask R-CNN and LETR Vision Transformer

TL;DR

This paper presents a novel computer vision pipeline combining Mask R-CNN and LETR transformer to accurately estimate leaf angles in field images, facilitating on-site plant phenotyping for crop improvement.

Contribution

The study introduces a new CV pipeline that integrates Mask R-CNN and LETR transformer for precise leaf angle estimation in real-world field conditions.

Findings

Achieved 0.98 cosine similarity with manual measurements.

Validated pipeline on two large field image datasets.

Demonstrated feasibility for on-site phenotyping applications.

Abstract

Modern day studies show a high degree of correlation between high yielding crop varieties and plants with upright leaf angles. It is observed that plants with upright leaf angles intercept more light than those without upright leaf angles, leading to a higher rate of photosynthesis. Plant scientists and breeders benefit from tools that can directly measure plant parameters in the field i.e. on-site phenotyping. The estimation of leaf angles by manual means in a field setting is tedious and cumbersome. We mitigate the tedium using a combination of the Mask R-CNN instance segmentation neural network, and Line Segment Transformer (LETR), a vision transformer. The proposed Computer Vision (CV) pipeline is applied on two image datasets, Summer 2015-Ames ULA and Summer 2015- Ames MLA, with a combined total of 1,827 plant images collected in the field using FieldBook, an Android application aimed at on-site phenotyping. The leaf angles estimated by the proposed pipeline on the image datasets are compared to two independent manual measurements using ImageJ, a Java-based image processing program developed at the National Institutes of Health and the Laboratory for Optical and Computational Instrumentation. The results, when compared for similarity using the Cosine Similarity measure, exhibit 0.98 similarity scores on both independent measurements of Summer 2015-Ames ULA and Summer 2015-Ames MLA image datasets, demonstrating the feasibility of the proposed pipeline for on-site measurement of leaf angles.