# Automated Food Weight and Content Estimation Using Computer Vision and AI Algorithms: Phase 2

## TL;DR

This paper presents an AI-based system using cameras to identify food items and estimate their weight in dining halls, achieving high accuracy.

## Contribution

A novel computer vision system for food content identification and weight estimation using RGB and depth cameras with calibrated density models.

## Key findings

- The YOLO-based model achieved a mean Average Precision (mAP) of 0.873 for food content identification.
- Food weight estimation for rice and chicken had error margins of 5.07% and 3.75%, respectively.
- Combining RGB and depth cameras with density models enabled accurate volume-to-weight conversion.

## Abstract

The work aims to leverage computer vision and artificial intelligence technologies to quantify key components in food catering services. Specifically, it focuses on content identification and portion size estimation in a dining hall setting, typical of corporate and educational settings. An RGB camera is employed to capture the tray delivery process in a self-service restaurant, providing test images for content identification algorithm comparison, using standard evaluation metrics. The approach utilizes the YOLO architecture, a widely recognized deep learning model for object detection and computer vision. The model is trained on labeled image data, and its performance is assessed using a precision–recall curve at a confidence threshold of 0.5, achieving a mean Average Precision (mAP) of 0.873, indicating robust overall performance. The weight estimation procedure combines computer vision techniques to measure food volume using both RGB and depth cameras. Subsequently, density models specific to each food type are applied to estimate the detected food weight. The estimation model’s parameters are calibrated through experiments that generate volume-to-weight conversion tables for different food items. Validation of the system was conducted using rice and chicken, yielding error margins of 5.07% and 3.75%, respectively, demonstrating the feasibility and accuracy of the proposed method.