Pinterest Canvas: Large-Scale Image Generation at Pinterest

TL;DR

Pinterest Canvas is a large-scale, adaptable image generation system that enables precise editing and enhancement for product-specific use cases, outperforming third-party models and improving user engagement.

Contribution

The paper introduces Pinterest Canvas, a scalable image generation framework with task-specific fine-tuning, tailored for strict product requirements and diverse downstream applications.

Findings

Achieved 18.0% and 12.5% engagement lift in online experiments.

Outperformed third-party models in human raters' evaluations.

Demonstrated versatility with multi-image synthesis and image-to-video tasks.

Abstract

While recent image generation models demonstrate a remarkable ability to handle a wide variety of image generation tasks, this flexibility makes them hard to control via prompting or simple inference adaptation alone, rendering them unsuitable for use cases with strict product requirements. In this paper, we introduce Pinterest Canvas, our large-scale image generation system built to support image editing and enhancement use cases at Pinterest. Canvas is first trained on a diverse, multimodal dataset to produce a foundational diffusion model with broad image-editing capabilities. However, rather than relying on one generic model to handle every downstream task, we instead rapidly fine-tune variants of this base model on task-specific datasets, producing specialized models for individual use cases. We describe key components of Canvas and summarize our best practices for dataset curation, training, and inference. We also showcase task-specific variants through case studies on background enhancement and aspect-ratio outpainting, highlighting how we tackle their specific product requirements. Online A/B experiments demonstrate that our enhanced images receive a significant 18.0% and 12.5% engagement lift, respectively, and comparisons with human raters further validate that our models outperform third-party models on these tasks. Finally, we showcase other Canvas variants, including multi-image scene synthesis and image-to-video generation, demonstrating that our approach can generalize to a wide variety of potential downstream tasks.