Prediction of Scene Plausibility

TL;DR

This paper introduces a new challenge for scene understanding algorithms to distinguish plausible from implausible scenes, emphasizing physical and functional plausibility, and evaluates models on a synthetic dataset.

Contribution

It defines scene plausibility as the probability of real-world occurrence, and creates a synthetic dataset to test neural networks' ability to recognize plausible scenes.

Findings

Neural networks can be trained to distinguish plausible from implausible scenes.

The synthetic dataset effectively evaluates scene plausibility recognition.

Physical and functional cues are crucial for plausibility assessment.

Abstract

Understanding the 3D world from 2D images involves more than detection and segmentation of the objects within the scene. It also includes the interpretation of the structure and arrangement of the scene elements. Such understanding is often rooted in recognizing the physical world and its limitations, and in prior knowledge as to how similar typical scenes are arranged. In this research we pose a new challenge for neural network (or other) scene understanding algorithms - can they distinguish between plausible and implausible scenes? Plausibility can be defined both in terms of physical properties and in terms of functional and typical arrangements. Hence, we define plausibility as the probability of encountering a given scene in the real physical world. We build a dataset of synthetic images containing both plausible and implausible scenes, and test the success of various vision models in the task of recognizing and understanding plausibility.