# Masks-to-Skeleton: Multi-View Mask-Based Tree Skeleton Extraction with 3D Gaussian Splatting

## TL;DR

This paper introduces a new method to extract tree skeletons from multi-view images by using segmentation masks and 3D Gaussian splatting, avoiding the need for noisy point clouds.

## Contribution

The novel mask-guided graph optimization framework bypasses point cloud processing and uses 3D Gaussian splatting for direct skeleton estimation.

## Key findings

- The method improves completeness and structural accuracy of tree skeletons compared to point-cloud-based methods.
- Experiments on both synthetic and real-world plants show consistent performance improvements.
- The use of a minimum spanning tree algorithm during optimization ensures a valid tree structure.

## Abstract

Accurately reconstructing tree skeletons from multi-view images is challenging. While most existing works use skeletonization from 3D point clouds, thin branches with low-texture contrast often involve multi-view stereo (MVS) to produce noisy and fragmented point clouds, which break branch connectivity. Leveraging the recent development in accurate mask extraction from images, we introduce a mask-guided graph optimization framework that estimates a 3D skeleton directly from multi-view segmentation masks, bypassing the reliance on point cloud quality. In our method, a skeleton is modeled as a graph whose nodes store positions and radii while its adjacency matrix encodes branch connectivity. We use 3D Gaussian splatting (3DGS) to render silhouettes of the graph and directly optimize the nodes and the adjacency matrix to fit given multi-view silhouettes in a differentiable manner. Furthermore, we use a minimum spanning tree (MST) algorithm during the optimization loop to regularize the graph to a tree structure. Experiments on synthetic and real-world plants show consistent improvements in completeness and structural accuracy over existing point-cloud-based and heuristic baseline methods.