SlotPi: Physics-informed Object-centric Reasoning Models

TL;DR

SlotPi is a physics-informed, object-centric reasoning model that integrates physical laws with dynamic prediction, demonstrating strong adaptability and performance in diverse visual reasoning and fluid dynamics tasks.

Contribution

The paper introduces SlotPi, a novel model combining Hamiltonian physics with object-centric reasoning for improved dynamic simulation and reasoning.

Findings

Effective in prediction and VQA tasks on benchmark datasets

Validated on real-world fluid and object interaction datasets

Demonstrates strong adaptability across diverse scenarios

Abstract

Understanding and reasoning about dynamics governed by physical laws through visual observation, akin to human capabilities in the real world, poses significant challenges. Currently, object-centric dynamic simulation methods, which emulate human behavior, have achieved notable progress but overlook two critical aspects: 1) the integration of physical knowledge into models. Humans gain physical insights by observing the world and apply this knowledge to accurately reason about various dynamic scenarios; 2) the validation of model adaptability across diverse scenarios. Real-world dynamics, especially those involving fluids and objects, demand models that not only capture object interactions but also simulate fluid flow characteristics. To address these gaps, we introduce SlotPi, a slot-based physics-informed object-centric reasoning model. SlotPi integrates a physical module based on Hamiltonian principles with a spatio-temporal prediction module for dynamic forecasting. Our experiments highlight the model's strengths in tasks such as prediction and Visual Question Answering (VQA) on benchmark and fluid datasets. Furthermore, we have created a real-world dataset encompassing object interactions, fluid dynamics, and fluid-object interactions, on which we validated our model's capabilities. The model's robust performance across all datasets underscores its strong adaptability, laying a foundation for developing more advanced world models.