O2RNet: Occluder-Occludee Relational Network for Robust Apple Detection in Clustered Orchard Environments

TL;DR

This paper introduces O2RNet, a deep learning framework that models occluder-occludee relationships to improve apple detection accuracy in complex orchard environments with occlusions and clustering.

Contribution

The paper presents a novel occlusion-aware neural network with feature expansion for robust apple detection, outperforming existing models in accuracy and F1-score.

Findings

O2RNet achieves 94% detection accuracy.

O2RNet has an F1-score of 0.88.

The model effectively handles occlusions and clustering.

Abstract

Automated apple harvesting has attracted significant research interest in recent years due to its potential to revolutionize the apple industry, addressing the issues of shortage and high costs in labor. One key technology to fully enable efficient automated harvesting is accurate and robust apple detection, which is challenging due to complex orchard environments that involve varying lighting conditions and foliage/branch occlusions. Furthermore, clustered apples are common in the orchard, which brings additional challenges as the clustered apples may be identified as one apple. This will cause issues in localization for subsequent robotic operations. In this paper, we present the development of a novel deep learning-based apple detection framework, Occluder-Occludee Relational Network (O2RNet), for robust detection of apples in such clustered environments. This network exploits the occuluder-occludee relationship modeling head by introducing a feature expansion structure to enable the combination of layered traditional detectors to split clustered apples and foliage occlusions. More specifically, we collect a comprehensive apple orchard image dataset under different lighting conditions (overcast, front lighting, and back lighting) with frequent apple occlusions. We then develop a novel occlusion-aware network for apple detection, in which a feature expansion structure is incorporated into the convolutional neural networks to extract additional features generated by the original network for occluded apples. Comprehensive evaluations are performed, which show that the developed O2RNet outperforms state-of-the-art models with a higher accuracy of 94\% and a higher F1-score of 0.88 on apple detection.