Image Based Reconstruction of Liquids from 2D Surface Detections

TL;DR

This paper introduces a novel method for reconstructing liquids from 2D surface detections without prior knowledge of liquid properties, using particle-based optimization and dynamic prediction, validated on new datasets.

Contribution

It presents a new optimization framework for liquid reconstruction from surface masks, requiring no training data and incorporating dynamic prediction for improved accuracy.

Findings

Effective reconstruction of liquids from surface detections demonstrated in simulation.

Proposed method outperforms baseline approaches in accuracy.

Open source datasets facilitate further research in liquid reconstruction.

Abstract

In this work, we present a solution to the challenging problem of reconstructing liquids from image data. The challenges in reconstructing liquids, which is not faced in previous reconstruction works on rigid and deforming surfaces, lies in the inability to use depth sensing and color features due the variable index of refraction, opacity, and environmental reflections. Therefore, we limit ourselves to only surface detections (i.e. binary mask) of liquids as observations and do not assume any prior knowledge on the liquids properties. A novel optimization problem is posed which reconstructs the liquid as particles by minimizing the error between a rendered surface from the particles and the surface detections while satisfying liquid constraints. Our solvers to this optimization problem are presented and no training data is required to apply them. We also propose a dynamic prediction to seed the reconstruction optimization from the previous time-step. We test our proposed methods in simulation and on two new liquid datasets which we open source so the broader research community can continue developing in this under explored area.