Decision Quality Evaluation Framework at Pinterest

TL;DR

This paper presents a comprehensive, data-driven framework for evaluating moderation decision quality at Pinterest, leveraging expert-curated benchmarks and automated sampling to improve trustworthiness and policy management at scale.

Contribution

The paper introduces a novel Decision Quality Evaluation Framework utilizing a high-trust Golden Set and propensity score sampling to enhance decision assessment for content moderation.

Findings

Benchmarking LLM cost-performance trade-offs

Establishing data-driven prompt optimization methodology

Ensuring policy content integrity through continuous validation

Abstract

Online platforms require robust systems to enforce content safety policies at scale. A critical component of these systems is the ability to evaluate the quality of moderation decisions made by both human agents and Large Language Models (LLMs). However, this evaluation is challenging due to the inherent trade-offs between cost, scale, and trustworthiness, along with the complexity of evolving policies. To address this, we present a comprehensive Decision Quality Evaluation Framework developed and deployed at Pinterest. The framework is centered on a high-trust Golden Set (GDS) curated by subject matter experts (SMEs), which serves as a ground truth benchmark. We introduce an automated intelligent sampling pipeline that uses propensity scores to efficiently expand dataset coverage. We demonstrate the framework's practical application in several key areas: benchmarking the cost-performance trade-offs of various LLM agents, establishing a rigorous methodology for data-driven prompt optimization, managing complex policy evolution, and ensuring the integrity of policy content prevalence metrics via continuous validation. The framework enables a shift from subjective assessments to a data-driven and quantitative practice for managing content safety systems.