Can LLMs Configure Software Tools
Jai Kannan

TL;DR
This paper explores using Large-Language Models to improve software tool configuration by identifying initial conditions and narrowing search spaces, aiming to enhance efficiency and reproducibility in complex, high-dimensional settings.
Contribution
It introduces a novel approach leveraging LLMs like Chat-GPT for software configuration, addressing limitations of traditional methods and demonstrating potential benefits through experiments.
Findings
LLMs can identify effective starting conditions for configurations
Response caching and domain-specific keyword effects observed in LLM outputs
LLMs show promise in expediting hyperparameter optimization processes
Abstract
In software engineering, the meticulous configuration of software tools is crucial in ensuring optimal performance within intricate systems. However, the complexity inherent in selecting optimal configurations is exacerbated by the high-dimensional search spaces presented in modern applications. Conventional trial-and-error or intuition-driven methods are both inefficient and error-prone, impeding scalability and reproducibility. In this study, we embark on an exploration of leveraging Large-Language Models (LLMs) to streamline the software configuration process. We identify that the task of hyperparameter configuration for machine learning components within intelligent applications is particularly challenging due to the extensive search space and performance-critical nature. Existing methods, including Bayesian optimization, have limitations regarding initial setup, computational cost,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware Engineering Research · Machine Learning and Data Classification · Machine Learning in Materials Science
