Quantifying Daily Evolution of Mobile Software Based on Memory Allocator Churn

TL;DR

This paper introduces a novel method using user mode memory allocator churn to efficiently detect performance regressions in large-scale mobile software systems, providing a timely alternative to traditional profiling techniques.

Contribution

It proposes and prototypes a new approach leveraging memory allocator churn as a proxy metric for performance analysis in mobile software development.

Findings

Memory allocator churn provides deterministic measurements.

Churn analysis efficiently detects performance regressions.

It serves as a suitable alternative to task completion time measurement.

Abstract

The pace and volume of code churn necessary to evolve modern software systems present challenges for analyzing the performance impact of any set of code changes. Traditional methods used in performance analysis rely on extensive data collection and profiling, which often takes days. For large organizations utilizing Continuous Integration (CI) and Continuous Deployment (CD), these traditional techniques often fail to provide timely and actionable data. A different impact analysis method that allows for more efficient detection of performance regressions is needed. We propose the utilization of user mode memory allocator churn as a novel approach to performance engineering. User mode allocator churn acts as a proxy metric to evaluate the relative change in the cost of specific tasks. We prototyped the memory allocation churn methodology while engaged in performance engineering for a major iOS application. We find that calculating and analyzing memory allocator churn (a) results in deterministic measurements, (b) is efficient for determining the presence of both individual performance regressions and general performance-related trends, and (c) is a suitable alternative to measuring the task completion time.