Hyper-Universal Policy Approximation: Learning to Generate Actions from a Single Image using Hypernets

TL;DR

This paper introduces Hyper-Universal Policy Approximators (HUPA), a hypernetwork-based method that generates adaptable, small policy networks from a single image, enabling agents to generalize actions to unseen environments efficiently.

Contribution

The paper presents a novel hypernetwork approach for generating environment- and task-specific policies from a single image, improving generalization and efficiency for edge devices.

Findings

HUPA outperforms embedding-based methods in size-constrained policy generation.

HUPA demonstrates strong generalization to unseen environments in navigation tasks.

The approach effectively learns policies from minimal visual input.

Abstract

Inspired by Gibson's notion of object affordances in human vision, we ask the question: how can an agent learn to predict an entire action policy for a novel object or environment given only a single glimpse? To tackle this problem, we introduce the concept of Universal Policy Functions (UPFs) which are state-to-action mappings that generalize not only to new goals but most importantly to novel, unseen environments. Specifically, we consider the problem of efficiently learning such policies for agents with limited computational and communication capacity, constraints that are frequently encountered in edge devices. We propose the Hyper-Universal Policy Approximator (HUPA), a hypernetwork-based model to generate small task- and environment-conditional policy networks from a single image, with good generalization properties. Our results show that HUPAs significantly outperform an embedding-based alternative for generated policies that are size-constrained. Although this work is restricted to a simple map-based navigation task, future work includes applying the principles behind HUPAs to learning more general affordances for objects and environments.