Eating Smart: Advancing Health Informatics with the Grounding DINO based Dietary Assistant App

TL;DR

This paper introduces a smart dietary assistant app that uses advanced machine learning, specifically the Grounding DINO model, for accurate food detection and personalized health advice, aiming to improve dietary management for conditions like diabetes.

Contribution

It presents a novel application of the Grounding DINO model in a cross-platform dietary assistant app with real-time food recognition and personalized health insights.

Findings

Achieved an AP score of 52.5 on COCO dataset, indicating high food detection accuracy.

Developed a multi-platform app with real-time food scanning and personalized nutrition features.

Enhanced user privacy through self-hosted database integration.

Abstract

The Smart Dietary Assistant utilizes Machine Learning to provide personalized dietary advice, focusing on users with conditions like diabetes. This app leverages the Grounding DINO model, which combines a text encoder and image backbone to enhance food item detection without requiring a labeled dataset. With an AP score of 52.5 on the COCO dataset, the model demonstrates high accuracy in real-world scenarios, utilizing attention mechanisms to precisely recognize objects based on user-provided labels and images. Developed using React Native and TypeScript, the app operates seamlessly across multiple platforms and integrates a self-hosted PostgreSQL database, ensuring data integrity and enhancing user privacy. Key functionalities include personalized nutrition profiles, real-time food scanning, and health insights, facilitating informed dietary choices for health management and lifestyle optimization. Future developments aim to integrate wearable technologies for more tailored health recommendations. Keywords: Food Image Recognition, Machine Learning in Nutrition, Zero-Shot Object Detection