Food Ingredients Recognition through Multi-label Learning

TL;DR

This paper presents a multi-label deep learning framework for recognizing multiple food ingredients in images, benchmarking various neural network architectures and decoding schemes to establish a baseline for automated diet assessment.

Contribution

It introduces a comprehensive multi-label learning approach with multiple neural network architectures and decoding methods for food ingredient recognition.

Findings

Deep learning models can effectively detect multiple ingredients in food images.

Attention-based decoding schemes improve ingredient recognition accuracy.

Established a baseline on the Nutrition5K dataset for future research.

Abstract

The ability to recognize various food-items in a generic food plate is a key determinant for an automated diet assessment system. This study motivates the need for automated diet assessment and proposes a framework to achieve this. Within this framework, we focus on one of the core functionalities to visually recognize various ingredients. To this end, we employed a deep multi-label learning approach and evaluated several state-of-the-art neural networks for their ability to detect an arbitrary number of ingredients in a dish image. The models evaluated in this work follow a definite meta-structure, consisting of an encoder and a decoder component. Two distinct decoding schemes, one based on global average pooling and the other on attention mechanism, are evaluated and benchmarked. Whereas for encoding, several well-known architectures, including DenseNet, EfficientNet, MobileNet, Inception and Xception, were employed. We present promising preliminary results for deep learning-based ingredients detection, using a challenging dataset, Nutrition5K, and establish a strong baseline for future explorations.