A Performance Analyzer for a Public Cloud's ML-Augmented VM Allocator

TL;DR

SANJESH is a bi-level optimization tool that systematically stress-tests ML models in cloud VM allocators, revealing adverse interactions and worst-case scenarios not detectable by traditional methods.

Contribution

It introduces a probabilistic adversarial analyzer that uncovers harmful model interactions in cloud VM placement pipelines, improving robustness testing.

Findings

Uncovered scenarios causing 4x worse performance than existing evaluators.

Demonstrated effectiveness on production traces from a cloud operator.

Revealed complex model interactions that degrade cloud performance.

Abstract

Cloud operators increasingly deploy multiple ML models in their VM allocation pipelines. In such settings, individually benign predictions can shift and compound, severely degrading performance. In a cloud provider's VM placement pipeline, CPU, memory, and lifetime prediction models jointly determine server count, live migration frequency, and network utilization; yet no existing approach can systematically stress-test how these models adversely interact. Deterministic adversarial analyzers cannot capture probabilistic ML behavior, so operators miss failures that arise only from correlated distributional shifts across models In SANJESH, we formulate a bi-level optimization that captures how the ML models behave statistically and uncovers how they adversely interact. The outer level searches over what predictions the ML models could produce under distributional uncertainty to find adversarial conditions; the inner level evaluates how the VM allocator behaves given those predictions. When we applied it to the operator's production traces, SANJESH uncovered scenarios that cause $4\times$ worse performance than the operators' evaluator detected.