3D shape sensing and deep learning-based segmentation of strawberries

TL;DR

This paper evaluates modern 3D sensing technologies for agricultural shape perception and introduces a novel deep neural network for segmenting soft fruits based on shape, demonstrating superior performance over existing methods.

Contribution

The paper proposes a new 3D deep neural network architecture tailored for agricultural segmentation, leveraging organized 3D sensor data for improved accuracy and efficiency.

Findings

3D sensing improves object localization and shape estimation in agriculture.

The proposed neural network outperforms state-of-the-art 3D segmentation models.

Simulated results show potential for wider adoption of 3D perception in farming.

Abstract

Automation and robotisation of the agricultural sector are seen as a viable solution to socio-economic challenges faced by this industry. This technology often relies on intelligent perception systems providing information about crops, plants and the entire environment. The challenges faced by traditional 2D vision systems can be addressed by modern 3D vision systems which enable straightforward localisation of objects, size and shape estimation, or handling of occlusions. So far, the use of 3D sensing was mainly limited to indoor or structured environments. In this paper, we evaluate modern sensing technologies including stereo and time-of-flight cameras for 3D perception of shape in agriculture and study their usability for segmenting out soft fruit from background based on their shape. To that end, we propose a novel 3D deep neural network which exploits the organised nature of information originating from the camera-based 3D sensors. We demonstrate the superior performance and efficiency of the proposed architecture compared to the state-of-the-art 3D networks. Through a simulated study, we also show the potential of the 3D sensing paradigm for object segmentation in agriculture and provide insights and analysis of what shape quality is needed and expected for further analysis of crops. The results of this work should encourage researchers and companies to develop more accurate and robust 3D sensing technologies to assure their wider adoption in practical agricultural applications.