Segment Anything for comprehensive analysis of grapevine cluster architecture and berry properties

TL;DR

This study applies the Segment Anything Model (SAM) to automatically segment grapevine berries in images, achieving high accuracy and enabling detailed analysis of cluster architecture and compactness with minimal training.

Contribution

It demonstrates SAM's out-of-the-box effectiveness for grapevine berry segmentation, providing a scalable, accurate method for analyzing complex cluster traits without extensive model training.

Findings

SAM achieved Pearson r2=0.96 correlation with human berry counts.

Linear regression adjusted for visibility issues with R2=0.87.

Cluster imaging angle significantly affects berry count accuracy.

Abstract

Grape cluster architecture and compactness are complex traits influencing disease susceptibility, fruit quality, and yield. Evaluation methods for these traits include visual scoring, manual methodologies, and computer vision, with the latter being the most scalable approach. Most of the existing computer vision approaches for processing cluster images often rely on conventional segmentation or machine learning with extensive training and limited generalization. The Segment Anything Model (SAM), a novel foundation model trained on a massive image dataset, enables automated object segmentation without additional training. This study demonstrates out-of-the-box SAM's high accuracy in identifying individual berries in 2D cluster images. Using this model, we managed to segment approximately 3,500 cluster images, generating over 150,000 berry masks, each linked with spatial coordinates within their clusters. The correlation between human-identified berries and SAM predictions was very strong (Pearson r2=0.96). Although the visible berry count in images typically underestimates the actual cluster berry count due to visibility issues, we demonstrated that this discrepancy could be adjusted using a linear regression model (adjusted R2=0.87). We emphasized the critical importance of the angle at which the cluster is imaged, noting its substantial effect on berry counts and architecture. We proposed different approaches in which berry location information facilitated the calculation of complex features related to cluster architecture and compactness. Finally, we discussed SAM's potential integration into currently available pipelines for image generation and processing in vineyard conditions.