Collaborative Feature Learning from Social Media

TL;DR

This paper introduces a novel, label-free image feature learning approach using social media user behavior data, which outperforms traditional supervised methods in image similarity tasks and performs well on recognition benchmarks.

Contribution

It proposes a new data-driven feature learning paradigm that leverages social media user behavior data instead of category labels, demonstrating its effectiveness on a large-scale dataset.

Findings

Learned features outperform state-of-the-art in image similarity

Features perform competitively on recognition benchmarks

Social media data effectively guides feature learning

Abstract

Image feature representation plays an essential role in image recognition and related tasks. The current state-of-the-art feature learning paradigm is supervised learning from labeled data. However, this paradigm requires large-scale category labels, which limits its applicability to domains where labels are hard to obtain. In this paper, we propose a new data-driven feature learning paradigm which does not rely on category labels. Instead, we learn from user behavior data collected on social media. Concretely, we use the image relationship discovered in the latent space from the user behavior data to guide the image feature learning. We collect a large-scale image and user behavior dataset from Behance.net. The dataset consists of 1.9 million images and over 300 million view records from 1.9 million users. We validate our feature learning paradigm on this dataset and find that the learned feature significantly outperforms the state-of-the-art image features in learning better image similarities. We also show that the learned feature performs competitively on various recognition benchmarks.