AutoScout: Structured Optimization for Automating ML System Configuration

TL;DR

AutoScout is a versatile system that automates ML configuration tuning by formulating it as a hybrid optimization problem, significantly improving training speed across various models and hardware.

Contribution

It introduces a hybrid optimization framework for ML system configuration that handles hierarchical dependencies and reduces profiling costs, outperforming existing heuristics.

Findings

Achieves 2.7-3.0× training speedup over expert tuning.

Effectively handles heterogeneous feature types and dependencies.

Reduces profiling cost through adaptive feature prioritization.

Abstract

Machine learning (ML) systems expose a rapidly expanding configuration space spanning model-parallelism strategies, communication optimizations, and low-level runtime parameters. End-to-end system efficiency is highly sensitive to these choices, yet identifying high-performance configurations is challenging due to heterogeneous feature types (e.g., sparse and dense parameters), conditional dependencies (e.g., valid execution parameters only under specific upstream decisions), and the high search (profiling) cost. Existing approaches either optimize a narrow subset of configuration dimensions or rely on ad-hoc heuristics that fail to generalize as configuration spaces continue to grow. We present AutoScout, a general-purpose systems configurator for ML training, fine-tuning, and inference. It formulates the system configuration as a mixed-discrete/continuous optimization problem with hierarchical dependencies and introduces a hybrid optimization framework that jointly refines sparse structural decisions and dense execution parameters. To reduce profiling cost, AutoScout adaptively prioritizes high-impact configuration features and ensembles simulators with varying fidelity. Across diverse models, hardware platforms, and deployment objectives, AutoScout consistently identifies high-performance configurations, achieving 2.7-3.0$\times$ training speedup over expert-tuned settings.