Industrial LLM-based Code Optimization under Regulation: A Mixture-of-Agents Approach

TL;DR

This paper introduces a Mixture-of-Agents approach for industrial code optimization using LLMs, demonstrating cost and time savings in regulated environments and providing deployment insights for different model types.

Contribution

It is the first to apply MoA to industrial code optimization with real-world codebases and compares its effectiveness against existing ensemble and individual LLM methods.

Findings

MoA achieves 14.3% to 22.2% cost savings with open-source models

MoA is 28.6% to 32.2% faster in optimization times

Ensembles outperform individual LLMs in regulated environments

Abstract

Recent advancements in Large Language Models (LLMs) for code optimization have enabled industrial platforms to automate software performance engineering at unprecedented scale and speed. Yet, organizations in regulated industries face strict constraints on which LLMs they can use - many cannot utilize commercial models due to data privacy regulations and compliance requirements, creating a significant challenge for achieving high-quality code optimization while maintaining cost-effectiveness. We address this by implementing a Mixture-of-Agents (MoA) approach that directly synthesizes code from multiple specialized LLMs, comparing it against TurinTech AI's vanilla Genetic Algorithm (GA)-based ensemble system and individual LLM optimizers using real-world industrial codebases. Our key contributions include: (1) First MoA application to industrial code optimization using real-world codebases; (2) Empirical evidence that MoA excels with open-source models, achieving 14.3% to 22.2% cost savings and 28.6% to 32.2% faster optimization times for regulated environments; (3) Deployment guidelines demonstrating GA's advantage with commercial models while both ensembles outperform individual LLMs; and (4) Real-world validation across 50 code snippets and seven LLM combinations, generating over 8,700 variants, addresses gaps in industrial LLM ensemble evaluation. This provides actionable guidance for organizations balancing regulatory compliance with optimization performance in production environments.