Semantics for 2D Rasterization

TL;DR

This paper introduces $b$Skia, a formal semantics for the Skia graphics library, enabling verification and optimization of rasterization code, resulting in significant speedups on real-world web graphics.

Contribution

It provides a formal, mechanized semantics for Skia, identifies sub-optimal patterns in Chrome, and develops a verified optimizer that improves rasterization performance.

Findings

18.7% average speedup on top 100 websites

Optimizer applies patterns in under 32 microseconds

Verified correctness of replacements in Lean

Abstract

Rasterization is the process of determining the color of every pixel drawn by an application. Powerful rasterization libraries like Skia, CoreGraphics, and Direct2D put exceptional effort into drawing, blending, and rendering efficiently. Yet applications are still hindered by the inefficient sequences of operations that they ask these libraries to perform. Even Google Chrome, a highly optimized program co-developed with the Skia rasterization library, still produces inefficient instruction sequences even on the top 100 most visited websites. The underlying reason for this inefficiency is that rasterization libraries have complex semantics and opaque and non-obvious execution models. To address this issue, we introduce $\mu$Skia, a formal semantics for the Skia 2D graphics library, and mechanize this semantics in Lean. $\mu$Skia covers language and graphics features like canvas state, the layer stack, blending, and color filters, and the semantics itself is split into three strata to separate concerns and enable extensibility. We then identify four patterns of sub-optimal Skia code produced by Google Chrome, and then write replacements for each pattern. $\mu$Skia allows us to verify the replacements are correct, including identifying numerous tricky side conditions. We then develop a high-performance Skia optimizer that applies these patterns to speed up rasterization. On 99 Skia programs gathered from the top 100 websites, this optimizer yields a speedup of 18.7% over Skia's most modern GPU backend, while taking at most 32 $\mu$s for optimization. The speedups persist across a variety of websites, Skia backends, and GPUs. To provide true, end-to-end verification, optimization traces produced by the optimizer are loaded back into the $\mu$Skia semantics and translation validated in Lean.