Semi-Automatic Crowdsourcing Tool for Online Food Image Collection and Annotation

TL;DR

This paper presents a semi-automatic system combining web crawling, automatic food detection, and crowdsourcing to efficiently collect and annotate large datasets of food images for improved dietary assessment models.

Contribution

It introduces an integrated semi-automatic framework for online food image collection and annotation, enhancing dataset creation for food intake estimation.

Findings

System effectively filters irrelevant images using automatic detection.

Crowdsourcing accelerates accurate food labeling.

Enables large-scale food image dataset development.

Abstract

Assessing dietary intake accurately remains an open and challenging research problem. In recent years, image-based approaches have been developed to automatically estimate food intake by capturing eat occasions with mobile devices and wearable cameras. To build a reliable machine-learning models that can automatically map pixels to calories, successful image-based systems need large collections of food images with high quality groundtruth labels to improve the learned models. In this paper, we introduce a semi-automatic system for online food image collection and annotation. Our system consists of a web crawler, an automatic food detection method and a web-based crowdsoucing tool. The web crawler is used to download large sets of online food images based on the given food labels. Since not all retrieved images contain foods, we introduce an automatic food detection method to remove irrelevant images. We designed a web-based crowdsourcing tool to assist the crowd or human annotators to locate and label all the foods in the images. The proposed semi-automatic online food image collection system can be used to build large food image datasets with groundtruth labels efficiently from scratch.