Occlusion Reasoning for Skeleton Extraction of Self-Occluded Tree Canopies

TL;DR

This paper introduces a novel method for extracting the skeleton of self-occluded tree canopies by combining instance segmentation with 3D likelihood mapping and path search, improving accuracy in occluded environments.

Contribution

The paper proposes a new approach that estimates unobserved tree structures using segmentation and 3D likelihood maps, advancing skeleton extraction under occlusion conditions.

Findings

Outperforms baseline methods in synthetic datasets

Effective in highly occluded scenes

Qualitative success on real-world data

Abstract

In this work, we present a method to extract the skeleton of a self-occluded tree canopy by estimating the unobserved structures of the tree. A tree skeleton compactly describes the topological structure and contains useful information such as branch geometry, positions and hierarchy. This can be critical to planning contact interactions for agricultural manipulation, yet is difficult to gain due to occlusion by leaves, fruits and other branches. Our method uses an instance segmentation network to detect visible trunk, branches, and twigs. Then, based on the observed tree structures, we build a custom 3D likelihood map in the form of an occupancy grid to hypothesize on the presence of occluded skeletons through a series of minimum cost path searches. We show that our method outperforms baseline methods in highly occluded scenes, demonstrated through a set of experiments on a synthetic tree dataset. Qualitative results are also presented on a real tree dataset collected from the field.