SeasonScapes: Learning Large-scale Re-lightable 3D Landscapes with Seasonal Variation from Sparse Webcams

TL;DR

This paper presents SeasonScapes, a framework and dataset for creating large-scale, re-lightable 3D landscapes with seasonal variations from sparse webcam images, enabling realistic rendering of natural changes over time.

Contribution

The paper introduces a novel dataset and a framework that combines 3D reconstruction, diffusion-based inpainting, and relighting to model seasonal landscape changes from sparse data.

Findings

Constructed a 3D landscape dataset with over 85,000 images across 13 seasons.

Developed a mesh inpainting method using conditional diffusion models for occlusion filling.

Enabled relighting of landscapes with realistic seasonal appearance changes.

Abstract

We introduce SeasonScapes framework and a the SeasonScapes dataset: Swiss Sparse-view Mountain Scenes with Seasonal Changes that covers over 50 km x 60 km, composed of more than 85,000 webcam images captured from 32 different locations across 13 timestamps throughout a full year. By projecting these timestamp-specific images onto a 3D mesh, we construct seasonal 3D landscapes that reflect natural appearance changes over time. To address occlusions and missing data, we leverage conditional diffusion models for image-guided inpainting directly on the mesh. The resulting completed meshes can be further relighted using standard physically-based renderer.