Physics-Informed Learning of Characteristic Trajectories for Smoke Reconstruction

TL;DR

This paper introduces Neural Characteristic Trajectory Fields, a physics-informed neural approach for reconstructing smoke and obstacles from sparse RGB videos, emphasizing long-term conservation and obstacle handling.

Contribution

It proposes a novel Eulerian neural field representation for modeling fluid trajectories, enabling efficient long-term physics supervision and scene reconstruction.

Findings

Effective long-term conservation in smoke reconstruction

Handling of occlusion and static-dynamic entanglements

Integration with NeRF for scene reconstruction

Abstract

We delve into the physics-informed neural reconstruction of smoke and obstacles through sparse-view RGB videos, tackling challenges arising from limited observation of complex dynamics. Existing physics-informed neural networks often emphasize short-term physics constraints, leaving the proper preservation of long-term conservation less explored. We introduce Neural Characteristic Trajectory Fields, a novel representation utilizing Eulerian neural fields to implicitly model Lagrangian fluid trajectories. This topology-free, auto-differentiable representation facilitates efficient flow map calculations between arbitrary frames as well as efficient velocity extraction via auto-differentiation. Consequently, it enables end-to-end supervision covering long-term conservation and short-term physics priors. Building on the representation, we propose physics-informed trajectory learning and integration into NeRF-based scene reconstruction. We enable advanced obstacle handling through self-supervised scene decomposition and seamless integrated boundary constraints. Our results showcase the ability to overcome challenges like occlusion uncertainty, density-color ambiguity, and static-dynamic entanglements. Code and sample tests are at \url{https://github.com/19reborn/PICT_smoke}.