Predicate Hierarchies Improve Few-Shot State Classification

TL;DR

This paper introduces PHIER, a method that uses predicate hierarchies and structured latent spaces to improve few-shot state classification in robotic environments, enabling better generalization with limited data.

Contribution

PHIER is the first approach to leverage predicate hierarchies and hyperbolic embeddings for few-shot state classification in robotics.

Findings

PHIER outperforms existing methods in few-shot and out-of-distribution scenarios.

PHIER demonstrates strong zero- and few-shot generalization from simulation to real-world tasks.

Predicate hierarchies significantly improve state classification accuracy with limited data.

Abstract

State classification of objects and their relations is core to many long-horizon tasks, particularly in robot planning and manipulation. However, the combinatorial explosion of possible object-predicate combinations, coupled with the need to adapt to novel real-world environments, makes it a desideratum for state classification models to generalize to novel queries with few examples. To this end, we propose PHIER, which leverages predicate hierarchies to generalize effectively in few-shot scenarios. PHIER uses an object-centric scene encoder, self-supervised losses that infer semantic relations between predicates, and a hyperbolic distance metric that captures hierarchical structure; it learns a structured latent space of image-predicate pairs that guides reasoning over state classification queries. We evaluate PHIER in the CALVIN and BEHAVIOR robotic environments and show that PHIER significantly outperforms existing methods in few-shot, out-of-distribution state classification, and demonstrates strong zero- and few-shot generalization from simulated to real-world tasks. Our results demonstrate that leveraging predicate hierarchies improves performance on state classification tasks with limited data.