MOSAIC: Multimodal Multistakeholder-aware Visual Art Recommendation

TL;DR

MOSAIC is a multimodal, multistakeholder-aware visual art recommendation system that balances user preferences with stakeholder interests using advanced AI models, improving diversity and relevance.

Contribution

The paper introduces MOSAIC, a novel approach combining CLIP and BLIP architectures with joint optimization for stakeholder-aware art recommendations.

Findings

Popularity significantly influences user preferences.

Representativeness has a minimal effect on user perception.

MOSAIC benefits multiple art stakeholders beyond visitors.

Abstract

Visual art (VA) recommendation is complex, as it has to consider the interests of users (e.g. museum visitors) and other stakeholders (e.g. museum curators). We study how to effectively account for key stakeholders in VA recommendations while also considering user-centred measures such as novelty, serendipity, and diversity. We propose MOSAIC, a novel multimodal multistakeholder-aware approach using state-of-the-art CLIP and BLIP backbone architectures and two joint optimisation objectives: popularity and representative selection of paintings across different categories. We conducted an offline evaluation using preferences elicited from 213 users followed by a user study with 100 crowdworkers. We found a strong effect of popularity, which was positively perceived by users, and a minimal effect of representativeness. MOSAIC's impact extends beyond visitors, benefiting various art stakeholders. Its user-centric approach has broader applicability, offering advancements for content recommendation across domains that require considering multiple stakeholders.