Probing Physics Knowledge Using Tools from Developmental Psychology
Luis Piloto, Ari Weinstein, Dhruva TB, Arun Ahuja, Mehdi Mirza, Greg, Wayne, David Amos, Chia-chun Hung, Matt Botvinick

TL;DR
This paper introduces a novel application of the violation of expectations (VOE) method from developmental psychology to evaluate and track intuitive physics understanding in artificial learning systems, using specially designed probe datasets.
Contribution
It adapts the VOE technique for AI systems, introduces new physics probe datasets inspired by developmental psychology, and demonstrates its utility in assessing physics knowledge in deep learning models.
Findings
VOE can effectively evaluate physics understanding in AI.
Baseline models show measurable responses to physics violations.
Probe datasets reveal strengths and gaps in current AI physics learning.
Abstract
In order to build agents with a rich understanding of their environment, one key objective is to endow them with a grasp of intuitive physics; an ability to reason about three-dimensional objects, their dynamic interactions, and responses to forces. While some work on this problem has taken the approach of building in components such as ready-made physics engines, other research aims to extract general physical concepts directly from sensory data. In the latter case, one challenge that arises is evaluating the learning system. Research on intuitive physics knowledge in children has long employed a violation of expectations (VOE) method to assess children's mastery of specific physical concepts. We take the novel step of applying this method to artificial learning systems. In addition to introducing the VOE technique, we describe a set of probe datasets inspired by classic test stimuli…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsScience Education and Pedagogy · Neuroscience, Education and Cognitive Function
