Indoor Future Person Localization from an Egocentric Wearable Camera

TL;DR

This paper introduces a new egocentric dataset with labeled person trajectories and poses, and proposes an LSTM-based model that effectively predicts future person locations and movements in indoor environments from wearable camera footage.

Contribution

The work presents a novel dataset with detailed annotations and a tailored LSTM-based framework for future person localization in egocentric videos, advancing indoor mobility assistance applications.

Findings

The proposed method outperforms baseline models in predicting person trajectories.

The dataset includes 8,250 clips with 13,817 labeled person bounding boxes and pose information.

The approach reliably predicts future locations in indoor egocentric videos.

Abstract

Accurate prediction of future person location and movement trajectory from an egocentric wearable camera can benefit a wide range of applications, such as assisting visually impaired people in navigation, and the development of mobility assistance for people with disability. In this work, a new egocentric dataset was constructed using a wearable camera, with 8,250 short clips of a targeted person either walking 1) toward, 2) away, or 3) across the camera wearer in indoor environments, or 4) staying still in the scene, and 13,817 person bounding boxes were manually labelled. Apart from the bounding boxes, the dataset also contains the estimated pose of the targeted person as well as the IMU signal of the wearable camera at each time point. An LSTM-based encoder-decoder framework was designed to predict the future location and movement trajectory of the targeted person in this egocentric setting. Extensive experiments have been conducted on the new dataset, and have shown that the proposed method is able to reliably and better predict future person location and trajectory in egocentric videos captured by the wearable camera compared to three baselines.