DepthScape: Authoring 2.5D Designs via Depth Estimation, Semantic Understanding, and Geometry Extraction

TL;DR

DepthScape is a collaborative system that simplifies creating 2.5D visual effects by integrating depth estimation, semantic understanding, and geometry extraction to enable intuitive editing of images with realistic 3D effects.

Contribution

We introduce DepthScape, a novel system combining monocular depth reconstruction and vision-language analysis for easy 2.5D design creation.

Findings

Effective 3D placement of design elements achieved

System validated with diverse users and professional images

Expert evaluation confirms high-quality results

Abstract

2.5D effects, such as occlusion and perspective foreshortening, enhance visual dynamics and realism by incorporating 3D depth cues into 2D designs. However, creating such effects remains challenging and labor-intensive due to the complexity of depth perception. We introduce DepthScape, a human-AI collaborative system that facilitates 2.5D effect creation by directly placing design elements into 3D reconstructions. Using monocular depth reconstruction, DepthScape transforms images into 3D reconstructions where visual contents are placed to automatically achieve realistic occlusion and perspective foreshortening. To further simplify 3D placement through a 2D viewport, DepthScape uses a vision-language model to analyze source images and extract key visual components as content anchors for direct manipulation editing. We evaluate DepthScape with nine participants of varying design backgrounds, confirming the effectiveness of our creation pipeline. We also test on 100 professional stock images to assess robustness, and conduct an expert evaluation that confirms the quality of DepthScape's results.