Personalized Food Image Classification: Benchmark Datasets and New Baseline

TL;DR

This paper introduces benchmark datasets and a novel deep learning framework for personalized food image classification, addressing the challenge of individual dietary pattern recognition in real-world scenarios.

Contribution

It provides the first personalized food image datasets and proposes a self-supervised learning approach leveraging temporal features for improved classification.

Findings

Enhanced classification accuracy on personalized datasets.

Benchmark datasets reflecting real-world dietary patterns.

Effective use of self-supervised learning with temporal features.

Abstract

Food image classification is a fundamental step of image-based dietary assessment, enabling automated nutrient analysis from food images. Many current methods employ deep neural networks to train on generic food image datasets that do not reflect the dynamism of real-life food consumption patterns, in which food images appear sequentially over time, reflecting the progression of what an individual consumes. Personalized food classification aims to address this problem by training a deep neural network using food images that reflect the consumption pattern of each individual. However, this problem is under-explored and there is a lack of benchmark datasets with individualized food consumption patterns due to the difficulty in data collection. In this work, we first introduce two benchmark personalized datasets including the Food101-Personal, which is created based on surveys of daily dietary patterns from participants in the real world, and the VFNPersonal, which is developed based on a dietary study. In addition, we propose a new framework for personalized food image classification by leveraging self-supervised learning and temporal image feature information. Our method is evaluated on both benchmark datasets and shows improved performance compared to existing works. The dataset has been made available at: https://skynet.ecn.purdue.edu/~pan161/dataset_personal.html