User Preferences Modeling and Learning for Pleasing Photo Collage Generation

TL;DR

This paper introduces a novel framework that learns user preferences for photo collage aesthetics through subjective experiments, resulting in a model that generates more pleasing collages aligned with diverse user tastes.

Contribution

It presents a new subjective-experiment-based approach to model and learn pleasantness criteria for photo collages, outperforming existing methods in user preference.

Findings

Learned model effectively encodes inter-user pleasantness preferences.

Model generalizes well to new datasets with different themes.

Collages generated by the model are preferred by most users.

Abstract

In this paper we consider how to automatically create pleasing photo collages created by placing a set of images on a limited canvas area. The task is formulated as an optimization problem. Differently from existing state-of-the-art approaches, we here exploit subjective experiments to model and learn pleasantness from user preferences. To this end, we design an experimental framework for the identification of the criteria that need to be taken into account to generate a pleasing photo collage. Five different thematic photo datasets are used to create collages using state-of-the-art criteria. A first subjective experiment where several subjects evaluated the collages, emphasizes that different criteria are involved in the subjective definition of pleasantness. We then identify new global and local criteria and design algorithms to quantify them. The relative importance of these criteria are automatically learned by exploiting the user preferences, and new collages are generated. To validate our framework, we performed several psycho-visual experiments involving different users. The results shows that the proposed framework allows to learn a novel computational model which effectively encodes an inter-user definition of pleasantness. The learned definition of pleasantness generalizes well to new photo datasets of different themes and sizes not used in the learning. Moreover, compared with two state of the art approaches, the collages created using our framework are preferred by the majority of the users.