Active Gaze Control for Foveal Scene Exploration

TL;DR

This paper presents a method for active gaze control in scene exploration that emulates human visual behavior, using deep learning and information theory to efficiently identify objects with minimal gaze shifts.

Contribution

The paper introduces a novel approach combining calibrated deep object detection, semantic mapping, and information-theoretic gaze planning for foveal scene exploration.

Findings

Increases detection F1-score by 2-3 percentage points over random gaze shifts.

Reduces the number of gaze shifts to one third to achieve similar detection performance.

Demonstrates effective scene exploration with fewer gaze movements.

Abstract

Active perception and foveal vision are the foundations of the human visual system. While foveal vision reduces the amount of information to process during a gaze fixation, active perception will change the gaze direction to the most promising parts of the visual field. We propose a methodology to emulate how humans and robots with foveal cameras would explore a scene, identifying the objects present in their surroundings with in least number of gaze shifts. Our approach is based on three key methods. First, we take an off-the-shelf deep object detector, pre-trained on a large dataset of regular images, and calibrate the classification outputs to the case of foveated images. Second, a body-centered semantic map, encoding the objects classifications and corresponding uncertainties, is sequentially updated with the calibrated detections, considering several data fusion techniques. Third, the next best gaze fixation point is determined based on information-theoretic metrics that aim at minimizing the overall expected uncertainty of the semantic map. When compared to the random selection of next gaze shifts, the proposed method achieves an increase in detection F1-score of 2-3 percentage points for the same number of gaze shifts and reduces to one third the number of required gaze shifts to attain similar performance.