Graph Representation for Face Analysis in Image Collections

TL;DR

This paper introduces an optimal graph-based representation for analyzing social interactions in image collections, utilizing connectivity scores derived from facial cues, and provides a GUI for visual exploration of social relations.

Contribution

It presents a novel graph representation based on connectivity scores from facial features and expressions, along with a GUI for social relation analysis in image collections.

Findings

Effective analysis of social interactions in diverse image collections

Graph visualization reveals social structures and relations

Applicable to social event and online image data

Abstract

Given an image collection of a social event with a huge number of pictures, it is very useful to have tools that can be used to analyze how the individuals --that are present in the collection-- interact with each other. In this paper, we propose an optimal graph representation that is based on the `connectivity' of them. The connectivity of a pair of subjects gives a score that represents how `connected' they are. It is estimated based on co-occurrence, closeness, facial expressions, and the orientation of the head when they are looking to each other. In our proposed graph, the nodes represent the subjects of the collection, and the edges correspond to their connectivities. The location of the nodes is estimated according to their connectivity (the closer the nodes, the more connected are the subjects). Finally, we developed a graphical user interface in which we can click onto the nodes (or the edges) to display the corresponding images of the collection in which the subject of the nodes (or the connected subjects) are present. We present relevant results by analyzing a wedding celebration, a sitcom video, a volleyball game and images extracted from Twitter given a hashtag. We believe that this tool can be very helpful to detect the existing social relations in an image collection.