Tree-D Fusion: Simulation-Ready Tree Dataset from Single Images with Diffusion Priors

TL;DR

This paper presents Tree D-fusion, a large dataset of 600,000 3D tree models generated from single images using diffusion priors, enabling realistic simulation-ready trees with detailed branching structures.

Contribution

It introduces a novel method combining diffusion priors and genus-conditioned shape reconstruction to create a large-scale, simulation-ready 3D tree dataset from 2D images.

Findings

Generated 600,000 3D tree models from images

Successfully reconstructed detailed branching structures

Enabled realistic simulation of trees for various applications

Abstract

We introduce Tree D-fusion, featuring the first collection of 600,000 environmentally aware, 3D simulation-ready tree models generated through Diffusion priors. Each reconstructed 3D tree model corresponds to an image from Google's Auto Arborist Dataset, comprising street view images and associated genus labels of trees across North America. Our method distills the scores of two tree-adapted diffusion models by utilizing text prompts to specify a tree genus, thus facilitating shape reconstruction. This process involves reconstructing a 3D tree envelope filled with point markers, which are subsequently utilized to estimate the tree's branching structure using the space colonization algorithm conditioned on a specified genus.