A Contextual Bandit Approach for Learning to Plan in Environments with Probabilistic Goal Configurations

TL;DR

This paper introduces a modular, context-aware approach for object-goal navigation that efficiently searches for both static and movable objects in indoor environments using a probabilistic model and a contextual bandit framework.

Contribution

It presents a novel modular framework employing a contextual bandit approach to learn the likelihood of object sightings, enabling efficient navigation to both static and movable objects.

Findings

High sample efficiency in simulated environments

Reliable object detection in real-world settings

Effective navigation to movable objects with probabilistic modeling

Abstract

Object-goal navigation (Object-nav) entails searching, recognizing and navigating to a target object. Object-nav has been extensively studied by the Embodied-AI community, but most solutions are often restricted to considering static objects (e.g., television, fridge, etc.). We propose a modular framework for object-nav that is able to efficiently search indoor environments for not just static objects but also movable objects (e.g. fruits, glasses, phones, etc.) that frequently change their positions due to human intervention. Our contextual-bandit agent efficiently explores the environment by showing optimism in the face of uncertainty and learns a model of the likelihood of spotting different objects from each navigable location. The likelihoods are used as rewards in a weighted minimum latency solver to deduce a trajectory for the robot. We evaluate our algorithms in two simulated environments and a real-world setting, to demonstrate high sample efficiency and reliability.