Proactive Detection and Calibration of Seasonal Advertisements with Multimodal Large Language Models

TL;DR

This paper introduces PDCaSA, a system for proactive seasonal ad detection using multimodal large language models, improving ad relevance and user satisfaction in large-scale industrial settings.

Contribution

The paper presents a comprehensive approach for detecting seasonal advertisements with multimodal LLMs, including guidelines, challenges, and a high-performing detection model.

Findings

Achieved 0.97 top F1 score on in-house benchmark

Provided detailed insights on data annotation and modeling challenges

Envisioned MLMs as versatile tools for knowledge distillation and system enhancement

Abstract

A myriad of factors affect large scale ads delivery systems and influence both user experience and revenue. One such factor is proactive detection and calibration of seasonal advertisements to help with increasing conversion and user satisfaction. In this paper, we present Proactive Detection and Calibration of Seasonal Advertisements (PDCaSA), a research problem that is of interest for the ads ranking and recommendation community, both in the industrial setting as well as in research. Our paper provides detailed guidelines from various angles of this problem tested in, and motivated by a large-scale industrial ads ranking system. We share our findings including the clear statement of the problem and its motivation rooted in real-world systems, evaluation metrics, and sheds lights to the existing challenges, lessons learned, and best practices of data annotation and machine learning modeling to tackle this problem. Lastly, we present a conclusive solution we took during this research exploration: to detect seasonality, we leveraged Multimodal LLMs (MLMs) which on our in-house benchmark achieved 0.97 top F1 score. Based on our findings, we envision MLMs as a teacher for knowledge distillation, a machine labeler, and a part of the ensembled and tiered seasonality detection system, which can empower ads ranking systems with enriched seasonal information.