FoodTrack: Estimating Handheld Food Portions with Egocentric Video

TL;DR

FoodTrack is a novel egocentric video framework that accurately measures hand-held food volume, overcoming occlusion issues and eliminating the need for gesture recognition or fixed bite size assumptions.

Contribution

It introduces a flexible, direct volume estimation method for handheld food using egocentric video, improving accuracy over previous approaches.

Findings

Achieves approximately 7.01% absolute percentage loss in food volume estimation.

Outperforms previous methods with a 16.40% error under less flexible conditions.

Demonstrates robustness to hand occlusions and varying camera angles.

Abstract

Accurately tracking food consumption is crucial for nutrition and health monitoring. Traditional approaches typically require specific camera angles, non-occluded images, or rely on gesture recognition to estimate intake, making assumptions about bite size rather than directly measuring food volume. We propose the FoodTrack framework for tracking and measuring the volume of hand-held food items using egocentric video which is robust to hand occlusions and flexible with varying camera and object poses. FoodTrack estimates food volume directly, without relying on intake gestures or fixed assumptions about bite size, offering a more accurate and adaptable solution for tracking food consumption. We achieve absolute percentage loss of approximately 7.01% on a handheld food object, improving upon a previous approach that achieved a 16.40% mean absolute percentage error in its best case, under less flexible conditions.