Long-tailed Food Classification

TL;DR

This paper introduces new long-tailed food classification datasets and proposes a two-phase framework combining undersampling, knowledge distillation, and data augmentation to improve classification performance in imbalanced food image datasets.

Contribution

The paper presents the first long-tailed food classification datasets and a novel 2-phase method to address class imbalance in food image recognition.

Findings

Improved accuracy over state-of-the-art methods on Food101-LT and VFN-LT datasets.

Effective handling of class imbalance through undersampling and oversampling techniques.

Demonstrated potential for real-world dietary assessment applications.

Abstract

Food classification serves as the basic step of image-based dietary assessment to predict the types of foods in each input image. However, food image predictions in a real world scenario are usually long-tail distributed among different food classes, which cause heavy class-imbalance problems and a restricted performance. In addition, none of the existing long-tailed classification methods focus on food data, which can be more challenging due to the lower inter-class and higher intra-class similarity among foods. In this work, we first introduce two new benchmark datasets for long-tailed food classification including Food101-LT and VFN-LT where the number of samples in VFN-LT exhibits the real world long-tailed food distribution. Then we propose a novel 2-Phase framework to address the problem of class-imbalance by (1) undersampling the head classes to remove redundant samples along with maintaining the learned information through knowledge distillation, and (2) oversampling the tail classes by performing visual-aware data augmentation. We show the effectiveness of our method by comparing with existing state-of-the-art long-tailed classification methods and show improved performance on both Food101-LT and VFN-LT benchmarks. The results demonstrate the potential to apply our method to related real life applications.