Unveiling Real-Life Effects of Online Photo Sharing

TL;DR

This paper presents LERVUP, a novel method that assesses the impact of visual content in online user profiles by combining crowdsourced ratings, object detection, and an attention mechanism to reveal real-life effects of data sharing.

Contribution

It introduces LERVUP, a new approach that learns to rate visual user profiles in specific situations, enhancing understanding of data sharing impacts in social networks.

Findings

LERVUP achieves a strong correlation with manual ratings.

The approach effectively identifies impactful visual objects.

Performance is validated across multiple real-life situations.

Abstract

Social networks give free access to their services in exchange for the right to exploit their users' data. Data sharing is done in an initial context which is chosen by the users. However, data are used by social networks and third parties in different contexts which are often not transparent. In order to unveil such usages, we propose an approach which focuses on the effects of data sharing in impactful real-life situations. Focus is put on visual content because of its strong influence in shaping online user profiles. The approach relies on three components: (1) a set of visual objects with associated situation impact ratings obtained by crowdsourcing, (2) a corresponding set of object detectors for mining users' photos and (3) a ground truth dataset made of 500 visual user profiles which are manually rated per situation. These components are combined in LERVUP, a method which learns to rate visual user profiles in each situation. LERVUP exploits a new image descriptor which aggregates object ratings and object detections at user level and an attention mechanism which boosts highly-rated objects to prevent them from being overwhelmed by low-rated ones. Performance is evaluated per situation by measuring the correlation between the automatic ranking of profile ratings and a manual ground truth. Results indicate that LERVUP is effective since a strong correlation of the two rankings is obtained. A practical implementation of the approach in a mobile app which raises user awareness about shared data usage is also discussed.