Combining Open-box Simulation and Importance Sampling for Tuning Large-Scale Recommenders

TL;DR

This paper introduces a hybrid tuning method called Simulator-Guided Importance Sampling (SGIS) that combines simulation and importance sampling to efficiently optimize large-scale recommender systems, reducing computational costs while maintaining accuracy.

Contribution

The paper presents SGIS, a novel hybrid approach that integrates open-box simulation with importance sampling for efficient large-scale recommender tuning.

Findings

SGIS reduces computational costs significantly.

SGIS maintains high accuracy in KPI estimation.

SGIS improves key performance indicators effectively.

Abstract

Growing scale of recommender systems require extensive tuning to respond to market dynamics and system changes. We address the challenge of tuning a large-scale ads recommendation platform with multiple continuous parameters influencing key performance indicators (KPIs). Traditional methods like open-box Monte Carlo simulators, while accurate, are computationally expensive due to the high cost of evaluating numerous parameter settings. To mitigate this, we propose a hybrid approach Simulator-Guided Importance Sampling (SGIS) that combines open-box simulation with importance sampling (IS). SGIS leverages the strengths of both techniques: it performs a coarse enumeration over the parameter space to identify promising initial settings and then uses IS to iteratively refine these settings. This approach significantly reduces computational costs while maintaining high accuracy in KPI estimation. We demonstrate the effectiveness of SGIS through simulations as well as real-world experiments, showing that it achieves substantial improvements in KPIs with lower computational overhead compared to traditional methods.