Migrating Existing Container Workload to Kubernetes -- LLM Based Approach and Evaluation

TL;DR

This paper evaluates the effectiveness of large language models in generating Kubernetes manifests from specifications, highlighting their strengths and limitations in accuracy and readability.

Contribution

It introduces a benchmarking method for assessing LLMs in synthesizing Kubernetes manifests using the Compose specification.

Findings

LLMs generally produce accurate Kubernetes manifests for standard specifications.

Readability comments are often omitted in LLM outputs.

Accuracy drops for atypical or unclear specifications.

Abstract

Although Kubernetes has become a widespread open-source system that automates the management of containerized applications, its complexity can be a significant barrier, particularly for application developers unfamiliar with it. One approach employs large language models (LLMs) to assist developers in generating Kubernetes manifests; however it is currently impossible to determine whether the output satisfies given specifications and is comprehensible. In this study, we proposed a benchmarking method for evaluating the effectiveness of LLMs in synthesizing manifests, using the Compose specification -- a standard widely adopted by application developers -- as input. The proposed benchmarking method revealed that LLMs generally produce accurate results that compensate for simple specification gaps. However, we also observed that inline comments for readability were often omitted, and completion accuracy was low for atypical inputs with unclear intentions.