Biomass phenotyping of oilseed rape through UAV multi-view oblique imaging with 3DGS and SAM model

TL;DR

This study combines 3D Gaussian Splatting and the Segment Anything Model with UAV multi-view imaging to accurately estimate oilseed rape biomass, outperforming traditional methods in complex field environments.

Contribution

It introduces a novel integration of 3DGS and SAM for precise 3D reconstruction and biomass estimation from UAV images, surpassing existing techniques in accuracy and efficiency.

Findings

3DGS achieved high accuracy with PSNR over 27 and 29.

SAM module attained an mIoU of 0.961 and F1-score of 0.980.

Point cloud volume model showed the highest biomass estimation accuracy with R2 of 0.976.

Abstract

Biomass estimation of oilseed rape is crucial for optimizing crop productivity and breeding strategies. While UAV-based imaging has advanced high-throughput phenotyping, current methods often rely on orthophoto images, which struggle with overlapping leaves and incomplete structural information in complex field environments. This study integrates 3D Gaussian Splatting (3DGS) with the Segment Anything Model (SAM) for precise 3D reconstruction and biomass estimation of oilseed rape. UAV multi-view oblique images from 36 angles were used to perform 3D reconstruction, with the SAM module enhancing point cloud segmentation. The segmented point clouds were then converted into point cloud volumes, which were fitted to ground-measured biomass using linear regression. The results showed that 3DGS (7k and 30k iterations) provided high accuracy, with peak signal-to-noise ratios (PSNR) of 27.43 and 29.53 and training times of 7 and 49 minutes, respectively. This performance exceeded that of structure from motion (SfM) and mipmap Neural Radiance Fields (Mip-NeRF), demonstrating superior efficiency. The SAM module achieved high segmentation accuracy, with a mean intersection over union (mIoU) of 0.961 and an F1-score of 0.980. Additionally, a comparison of biomass extraction models found the point cloud volume model to be the most accurate, with an determination coefficient (R2) of 0.976, root mean square error (RMSE) of 2.92 g/plant, and mean absolute percentage error (MAPE) of 6.81%, outperforming both the plot crop volume and individual crop volume models. This study highlights the potential of combining 3DGS with multi-view UAV imaging for improved biomass phenotyping.