Calorie Counter: RGB-Depth Visual Estimation of Energy Expenditure at Home

TL;DR

This paper introduces a vision-based system for estimating energy expenditure at home using RGB-D data, validated against gas exchange measurements, enabling remote health monitoring without wearable devices.

Contribution

It presents the first validated RGB-D vision framework for calorific estimation in daily living, introducing a new dataset and establishing a baseline for future research.

Findings

The vision system achieves accuracy surpassing traditional MET-based estimates.

The approach is pose-invariant and individual-independent, suitable for home environments.

The SPHERE-calorie dataset enables future benchmarking in this area.

Abstract

We present a new framework for vision-based estimation of calorific expenditure from RGB-D data - the first that is validated on physical gas exchange measurements and applied to daily living scenarios. Deriving a person's energy expenditure from sensors is an important tool in tracking physical activity levels for health and lifestyle monitoring. Most existing methods use metabolic lookup tables (METs) for a manual estimate or systems with inertial sensors which ultimately require users to wear devices. In contrast, the proposed pose-invariant and individual-independent vision framework allows for a remote estimation of calorific expenditure. We introduce, and evaluate our approach on, a new dataset called SPHERE-calorie, for which visual estimates can be compared against simultaneously obtained, indirect calorimetry measures based on gas exchange. % based on per breath gas exchange. We conclude from our experiments that the proposed vision pipeline is suitable for home monitoring in a controlled environment, with calorific expenditure estimates above accuracy levels of commonly used manual estimations via METs. With the dataset released, our work establishes a baseline for future research for this little-explored area of computer vision.