Automated Design Optimization via Strategic Search with Large Language Models
Anthony Carreon, Vansh Sharma, Venkat Raman

TL;DR
This paper introduces AUTO, an LLM-based framework for design optimization that uses strategic reasoning to efficiently explore complex, ill-defined search spaces, demonstrating competitive results and cost savings in GPU code optimization.
Contribution
The paper presents AUTO, a novel LLM-driven approach that treats design optimization as a gradient-free search, integrating strategic reasoning and collaborative agents for improved efficiency.
Findings
Achieves 50-70% search efficiency compared to Bayesian optimization.
Completes optimizations in about 8 hours at a lower cost than manual development.
Generates solutions competitive with expert implementations in GPU code optimization.
Abstract
Traditional optimization methods excel in well-defined search spaces but struggle with design problems where transformations and design parameters are difficult to define. Large language models (LLMs) offer a promising alternative by dynamically interpreting design spaces and leveraging encoded domain knowledge. To this end, we introduce AUTO, an LLM agent framework that treats design optimization as a gradient-free search problem guided by strategic LLM reasoning. The framework employs two collaborative agents: a Strategist that selects between exploration and exploitation strategies, and an Implementor that executes detailed designs. Applied to GPU code optimization -- a domain critical to fields from machine learning to scientific computing -- AUTO generates solutions competitive with expert implementations for chemical kinetics integration and dense matrix multiplication. The…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMachine Learning in Materials Science · Machine Learning and Data Classification · Advanced Multi-Objective Optimization Algorithms
