Kiwifruit detection in challenging conditions

TL;DR

This paper introduces a semantic segmentation method with novel image preprocessing techniques to improve kiwifruit detection accuracy in challenging orchard lighting conditions, addressing occlusion and lighting issues for robotic harvesting.

Contribution

The paper presents two new image preprocessing techniques that significantly enhance kiwifruit detection under harsh lighting in a semantic segmentation framework.

Findings

Semantic segmentation achieves 0.82 F1_score in typical lighting.

Preprocessing improves F1_score to 0.42 under harsh lighting.

Detects 87% of non-occluded and 30% of occluded kiwifruit.

Abstract

Accurate and reliable kiwifruit detection is one of the biggest challenges in developing a selective fruit harvesting robot. The vision system of an orchard robot faces difficulties such as dynamic lighting conditions and fruit occlusions. This paper presents a semantic segmentation approach with two novel image prepossessing techniques designed to detect kiwifruit under the harsh lighting conditions found in the canopy. The performance of the presented system is evaluated on a 3D real-world image set of kiwifruit under different lighting conditions (typical, glare, and overexposed). Alone the semantic segmentation approach achieves an F1_score of 0.82 on the typical lighting image set, but struggles with harsh lighting with an F1_score of 0.13. Utilising the prepossessing techniques the vision system under harsh lighting improves to an F1_score 0.42. To address the fruit occlusion challenge, the overall approach was found to be capable of detecting 87.0% of non-occluded and 30.0% of occluded kiwifruit across all lighting conditions.