Beyond Classification: Latent User Interests Profiling from Visual Contents Analysis

TL;DR

This paper introduces a novel approach to user preference profiling on image-centric social platforms by directly learning latent visual preferences from image contents using deep metric learning, revealing individual differences.

Contribution

It proposes a deep CNN-based distance metric learning method to extract and analyze fine-grained visual preferences directly from images, surpassing traditional classification methods.

Findings

Users have distinct, identifiable visual preferences within the same image category.

Visual preferences are consistent over users' lifetimes.

Finer-grained profiling enhances understanding of user preferences.

Abstract

User preference profiling is an important task in modern online social networks (OSN). With the proliferation of image-centric social platforms, such as Pinterest, visual contents have become one of the most informative data streams for understanding user preferences. Traditional approaches usually treat visual content analysis as a general classification problem where one or more labels are assigned to each image. Although such an approach simplifies the process of image analysis, it misses the rich context and visual cues that play an important role in people's perception of images. In this paper, we explore the possibilities of learning a user's latent visual preferences directly from image contents. We propose a distance metric learning method based on Deep Convolutional Neural Networks (CNN) to directly extract similarity information from visual contents and use the derived distance metric to mine individual users' fine-grained visual preferences. Through our preliminary experiments using data from 5,790 Pinterest users, we show that even for the images within the same category, each user possesses distinct and individually-identifiable visual preferences that are consistent over their lifetime. Our results underscore the untapped potential of finer-grained visual preference profiling in understanding users' preferences.