Understanding and Exploiting Object Interaction Landscapes

TL;DR

This paper introduces a novel, object-agnostic representation of physical interactions that captures dynamic object behaviors and functions, enabling better understanding and retrieval of objects based on their interaction landscapes.

Contribution

The work presents a new functional descriptor called the interaction landscape, which models object interactions in a spatio-temporal framework independent of interaction type.

Findings

Interaction landscapes effectively capture subtle dynamic effects.

The method enables shape correspondence based on functional key points.

Objects can be related and retrieved based on their interaction profiles.

Abstract

Interactions play a key role in understanding objects and scenes, for both virtual and real world agents. We introduce a new general representation for proximal interactions among physical objects that is agnostic to the type of objects or interaction involved. The representation is based on tracking particles on one of the participating objects and then observing them with sensors appropriately placed in the interaction volume or on the interaction surfaces. We show how to factorize these interaction descriptors and project them into a particular participating object so as to obtain a new functional descriptor for that object, its interaction landscape, capturing its observed use in a spatio-temporal framework. Interaction landscapes are independent of the particular interaction and capture subtle dynamic effects in how objects move and behave when in functional use. Our method relates objects based on their function, establishes correspondences between shapes based on functional key points and regions, and retrieves peer and partner objects with respect to an interaction.