Mining Discriminative Food Regions for Accurate Food Recognition

TL;DR

This paper introduces PAR-Net, a novel end-to-end network that mines discriminative food regions for improved food recognition, achieving state-of-the-art accuracy on multiple datasets including a new Sushi-50 dataset.

Contribution

The paper proposes a new architecture, PAR-Net, that leverages adversarially mined discriminative regions for enhanced food recognition accuracy.

Findings

Achieved top-1 accuracy of 90.4% on Food-101

Achieved top-1 accuracy of 90.2% on Vireo-172

Achieved top-1 accuracy of 92.0% on Sushi-50

Abstract

Automatic food recognition is the very first step towards passive dietary monitoring. In this paper, we address the problem of food recognition by mining discriminative food regions. Taking inspiration from Adversarial Erasing, a strategy that progressively discovers discriminative object regions for weakly supervised semantic segmentation, we propose a novel network architecture in which a primary network maintains the base accuracy of classifying an input image, an auxiliary network adversarially mines discriminative food regions, and a region network classifies the resulting mined regions. The global (the original input image) and the local (the mined regions) representations are then integrated for the final prediction. The proposed architecture denoted as PAR-Net is end-to-end trainable, and highlights discriminative regions in an online fashion. In addition, we introduce a new fine-grained food dataset named as Sushi-50, which consists of 50 different sushi categories. Extensive experiments have been conducted to evaluate the proposed approach. On three food datasets chosen (Food-101, Vireo-172, and Sushi-50), our approach performs consistently and achieves state-of-the-art results (top-1 testing accuracy of $90.4\%$, $90.2\%$, $92.0\%$, respectively) compared with other existing approaches. Dataset and code are available at https://github.com/Jianing-Qiu/PARNet