Ambig-IaC: Multi-level Disambiguation for Interactive Cloud Infrastructure-as-Code Synthesis

TL;DR

Ambig-IaC introduces a multi-level disambiguation framework for improving the synthesis of cloud infrastructure configurations from ambiguous natural language prompts, leveraging hierarchical structure and clarification questions.

Contribution

The paper presents a training-free, disagreement-driven approach for disambiguating IaC specifications, along with a new benchmark dataset and evaluation framework.

Findings

Our method outperforms baselines with +18.4% and +25.4% improvements on structure and attribute metrics.

The framework effectively narrows down ambiguous IaC prompts through targeted clarification questions.

Ambig-IaC demonstrates the importance of hierarchical disambiguation in complex code synthesis tasks.

Abstract

The scale and complexity of modern cloud infrastructure have made Infrastructure-as-Code (IaC) essential for managing deployments. While large Language models (LLMs) are increasingly being used to generate IaC configurations from natural language, user requests are often underspecified. Unlike traditional code generation, IaC configurations cannot be executed cheaply or iteratively repaired, forcing the LLMs into an almost one-shot regime. We observe that ambiguity in IaC exhibits a tractable compositional structure: configurations decompose into three hierarchical axes (resources, topology, attributes) where higher-level decisions constrain lower-level ones. We propose a training-free, disagreement-driven framework that generates diverse candidate specifications, identifies structural disagreements across these axes, ranks them by informativeness, and produces targeted clarification questions that progressively narrow the configuration space. We introduce \textsc{Ambig-IaC}, a benchmark of 300 validated IaC tasks with ambiguous prompts, and an evaluation framework based on graph edit distance and embedding similarity. Our method outperforms the strongest baseline, achieving relative improvements of +18.4\% and +25.4\% on structure and attribute evaluations, respectively.