Multi-Object Graph Affordance Network: Goal-Oriented Planning through Learned Compound Object Affordances

TL;DR

This paper introduces a graph neural network-based model that learns complex affordances of compound objects, enabling goal-oriented robot planning for stacking, inserting, and passing through objects in both simulated and real-world settings.

Contribution

The novel Multi-Object Graph Affordance Network models compound object affordances using graph neural networks, extending affordance learning to arbitrary object combinations for improved robot planning.

Findings

Successfully modeled affordances of complex compound objects.

Enabled goal-oriented planning for stacking and insertion tasks.

Outperformed baseline models in benchmarks.

Abstract

Learning object affordances is an effective tool in the field of robot learning. While the data-driven models investigate affordances of single or paired objects, there is a gap in the exploration of affordances of compound objects composed of an arbitrary number of objects. We propose the Multi-Object Graph Affordance Network which models complex compound object affordances by learning the outcomes of robot actions that facilitate interactions between an object and a compound. Given the depth images of the objects, the object features are extracted via convolution operations and encoded in the nodes of graph neural networks. Graph convolution operations are used to encode the state of the compounds, which are used as input to decoders to predict the outcome of the object-compound interactions. After learning the compound object affordances, given different tasks, the learned outcome predictors are used to plan sequences of stack actions that involve stacking objects on top of each other, inserting smaller objects into larger containers and passing through ring-like objects through poles. We showed that our system successfully modeled the affordances of compound objects that include concave and convex objects, in both simulated and real-world environments. We benchmarked our system with a baseline model to highlight its advantages.