Critic Guided Segmentation of Rewarding Objects in First-Person Views

TL;DR

This paper introduces a critic-guided segmentation method that learns to identify rewarding objects in first-person images using sparse reward signals, achieving state-of-the-art results in a complex 3D environment.

Contribution

It presents a novel approach that trains a segmentation network solely with critic feedback, without explicit object annotations, for identifying rewarding objects in complex scenes.

Findings

Achieved first place in the NeurIPS 2020 MineRL competition.

Successfully learned to segment rewarding objects in 3D environments.

Demonstrated effectiveness with sparse reward signals.

Abstract

This work discusses a learning approach to mask rewarding objects in images using sparse reward signals from an imitation learning dataset. For that, we train an Hourglass network using only feedback from a critic model. The Hourglass network learns to produce a mask to decrease the critic's score of a high score image and increase the critic's score of a low score image by swapping the masked areas between these two images. We trained the model on an imitation learning dataset from the NeurIPS 2020 MineRL Competition Track, where our model learned to mask rewarding objects in a complex interactive 3D environment with a sparse reward signal. This approach was part of the 1st place winning solution in this competition. Video demonstration and code: https://rebrand.ly/critic-guided-segmentation