Inferring the Future by Imagining the Past
Kartik Chandra, Tony Chen, Tzu-Mao Li, Jonathan Ragan-Kelley, Josh, Tenenbaum

TL;DR
This paper presents a Monte Carlo algorithm inspired by computer graphics to model how humans infer past and future events from static images, aligning well with human intuition across various domains.
Contribution
It introduces a novel Monte Carlo inference method based on path tracing concepts, bridging cognitive science and computer graphics to explain human scene understanding.
Findings
Algorithm correlates with human judgments across domains
Uses a small, cognitively plausible number of samples
Connects Monte Carlo path tracing with theory of mind inference
Abstract
A single panel of a comic book can say a lot: it can depict not only where the characters currently are, but also their motions, their motivations, their emotions, and what they might do next. More generally, humans routinely infer complex sequences of past and future events from a *static snapshot* of a *dynamic scene*, even in situations they have never seen before. In this paper, we model how humans make such rapid and flexible inferences. Building on a long line of work in cognitive science, we offer a Monte Carlo algorithm whose inferences correlate well with human intuitions in a wide variety of domains, while only using a small, cognitively-plausible number of samples. Our key technical insight is a surprising connection between our inference problem and Monte Carlo path tracing, which allows us to apply decades of ideas from the computer graphics community to this…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsArtificial Intelligence in Games · Advanced Image and Video Retrieval Techniques · Reinforcement Learning in Robotics
