CropNeRF: A Neural Radiance Field-Based Framework for Crop Counting

TL;DR

CropNeRF introduces a 3D instance segmentation framework using neural radiance fields and multi-view images to accurately count crops in complex outdoor environments, overcoming occlusion and ambiguity issues.

Contribution

The paper presents a novel 3D crop counting method leveraging NeRF-based view synthesis, crop visibility, and mask consistency scores, eliminating the need for crop-specific tuning.

Findings

Consistently accurate crop counts across diverse datasets

Superior performance compared to existing methods

Robustness to variations in crop appearance

Abstract

Rigorous crop counting is crucial for effective agricultural management and informed intervention strategies. However, in outdoor field environments, partial occlusions combined with inherent ambiguity in distinguishing clustered crops from individual viewpoints poses an immense challenge for image-based segmentation methods. To address these problems, we introduce a novel crop counting framework designed for exact enumeration via 3D instance segmentation. Our approach utilizes 2D images captured from multiple viewpoints and associates independent instance masks for neural radiance field (NeRF) view synthesis. We introduce crop visibility and mask consistency scores, which are incorporated alongside 3D information from a NeRF model. This results in an effective segmentation of crop instances in 3D and highly-accurate crop counts. Furthermore, our method eliminates the dependence on crop-specific parameter tuning. We validate our framework on three agricultural datasets consisting of cotton bolls, apples, and pears, and demonstrate consistent counting performance despite major variations in crop color, shape, and size. A comparative analysis against the state of the art highlights superior performance on crop counting tasks. Lastly, we contribute a cotton plant dataset to advance further research on this topic.