SG-DOR: Learning Scene Graphs with Direction-Conditioned Occlusion Reasoning for Pepper Plants

TL;DR

This paper introduces SG-DOR, a novel framework that uses scene graphs with direction-conditioned occlusion reasoning to improve robotic harvesting by accurately modeling organ relations and occlusions in dense crop canopies.

Contribution

SG-DOR is the first to incorporate direction-conditioned occlusion reasoning into scene graphs for robotic harvesting in dense crop environments.

Findings

Improved occlusion prediction with F1=0.73 and NDCG@3=0.85

Enhanced attachment inference with edge F1=0.83

Effective structured relational signals for intervention planning

Abstract

Robotic harvesting in dense crop canopies requires effective interventions that depend not only on geometry, but also on explicit, direction-conditioned relations identifying which organs obstruct a target fruit. We present SG-DOR (Scene Graphs with Direction-Conditioned Occlusion Reasoning), a relational framework that, given instance-segmented organ point clouds, infers a scene graph encoding physical attachments and direction-conditioned occlusion. We introduce an occlusion ranking task for retrieving and ranking candidate leaves for a target fruit and approach direction, and propose a direction-aware graph neural architecture with per-fruit leaf-set attention and union-level aggregation. Experiments on a multi-plant synthetic pepper dataset show improved occlusion prediction (F1=0.73, NDCG@3=0.85) and attachment inference (edge F1=0.83) over strong ablations, yielding a structured relational signal for downstream intervention planning.