Towards Bottom-Up Analysis of Social Food

TL;DR

This paper presents a large-scale analysis of Instagram food images, demonstrating that it is possible to recognize food categories with over 70% accuracy using only social media data and hashtags, without manual labeling.

Contribution

It introduces a data-driven method for recognizing food content in Instagram images by leveraging social network data and hashtags, achieving high accuracy without manual annotation.

Findings

Food recognition accuracy exceeds 70% for popular categories.

Social hashtags serve as noisy labels for image content.

The approach highlights potential for large-scale, automated social media content analysis.

Abstract

Social media provide a wealth of information for research into public health by providing a rich mix of personal data, location, hashtags, and social network information. Among these, Instagram has been recently the subject of many computational social science studies. However despite Instagram's focus on image sharing, most studies have exclusively focused on the hashtag and social network structure. In this paper we perform the first large scale content analysis of Instagram posts, addressing both the image and the associated hashtags, aiming to understand the content of partially-labelled images taken in-the-wild and the relationship with hashtags that individuals use as noisy labels. In particular, we explore the possibility of learning to recognise food image content in a data driven way, discovering both the categories of food, and how to recognise them, purely from social network data. Notably, we demonstrate that our approach to food recognition can often achieve accuracies greater than 70% in recognising popular food-related image categories, despite using no manual annotation. We highlight the current capabilities and future challenges and opportunities for such data-driven analysis of image content and the relation to hashtags.