Perceptual Evaluation of Liquid Simulation Methods

TL;DR

This paper introduces a new perceptual evaluation framework for liquid simulation methods using crowd-sourced user studies and reference videos to ensure consistent and reliable assessments of visual accuracy.

Contribution

It presents a novel evaluation approach leveraging reference videos and crowd-sourcing, enabling robust perceptual comparison of fluid simulation techniques.

Findings

Reference videos improve evaluation consistency

Crowd-sourced opinions effectively assess visual accuracy

The framework can evaluate various simulation methods

Abstract

This paper proposes a novel framework to evaluate fluid simulation methods based on crowd-sourced user studies in order to robustly gather large numbers of opinions. The key idea for a robust and reliable evaluation is to use a reference video from a carefully selected real-world setup in the user study. By conducting a series of controlled user studies and comparing their evaluation results, we observe various factors that affect the perceptual evaluation. Our data show that the availability of a reference video makes the evaluation consistent. We introduce this approach for computing scores of simulation methods as visual accuracy metric. As an application of the proposed framework, a variety of popular simulation methods are evaluated.