How Much You Ate? Food Portion Estimation on Spoons

TL;DR

This paper presents a novel camera-based system that tracks food on utensils from a fixed user-facing perspective, enabling more accurate and convenient dietary intake monitoring without requiring multiple images or changing camera angles.

Contribution

The study introduces a stationary, user-facing camera system that tracks food on utensils, improving accuracy and convenience over existing image-based methods.

Findings

Effective tracking of food on utensils demonstrated

High accuracy in estimating nutritional content of soups and stews

Non-invasive and user-friendly monitoring approach

Abstract

Monitoring dietary intake is a crucial aspect of promoting healthy living. In recent years, advances in computer vision technology have facilitated dietary intake monitoring through the use of images and depth cameras. However, the current state-of-the-art image-based food portion estimation algorithms assume that users take images of their meals one or two times, which can be inconvenient and fail to capture food items that are not visible from a top-down perspective, such as ingredients submerged in a stew. To address these limitations, we introduce an innovative solution that utilizes stationary user-facing cameras to track food items on utensils, not requiring any change of camera perspective after installation. The shallow depth of utensils provides a more favorable angle for capturing food items, and tracking them on the utensil's surface offers a significantly more accurate estimation of dietary intake without the need for post-meal image capture. The system is reliable for estimation of nutritional content of liquid-solid heterogeneous mixtures such as soups and stews. Through a series of experiments, we demonstrate the exceptional potential of our method as a non-invasive, user-friendly, and highly accurate dietary intake monitoring tool.