Large-Scale Metric Computation in Online Controlled Experiment Platform
Tao Xiong, Yong Wang

TL;DR
This paper presents an efficient large-scale metric computation method using bit-sliced index arithmetic for online controlled experiments, demonstrated in WeChat's platform, significantly improving performance.
Contribution
It introduces the application of bit-sliced index arithmetic for scalable metric computation in online experiment platforms, with real-world implementation and performance validation.
Findings
BSI arithmetic significantly improves computation efficiency
The approach scales well with large experiment data
Implementation in WeChat demonstrates practical effectiveness
Abstract
Online controlled experiment (also called A/B test or experiment) is the most important tool for decision-making at a wide range of data-driven companies like Microsoft, Google, Meta, etc. Metric computation is the core procedure for reaching a conclusion during an experiment. With the growth of experiments and metrics in an experiment platform, computing metrics efficiently at scale becomes a non-trivial challenge. This work shows how metric computation in WeChat experiment platform can be done efficiently using bit-sliced index (BSI) arithmetic. This approach has been implemented in a real world system and the performance results are presented, showing that the BSI arithmetic approach is very suitable for large-scale metric computation scenarios.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
