Analyzing and Mitigating (with LLMs) the Security Misconfigurations of Helm Charts from Artifact Hub

TL;DR

This paper evaluates the security of Helm charts from Artifact Hub by comparing existing tools' misconfiguration detection with LLM-based mitigation suggestions, and assesses false positives through manual analysis.

Contribution

It introduces a pipeline combining chart analysis tools and LLMs to detect, mitigate, and verify security misconfigurations in Helm charts from Artifact Hub.

Findings

LLMs can effectively suggest mitigations for misconfigurations.

Existing tools report varying misconfigurations with some false positives.

Refactored charts often meet security policies after LLM mitigation.

Abstract

Background: Helm is a package manager that allows defining, installing, and upgrading applications with Kubernetes (K8s), a popular container orchestration platform. A Helm chart is a collection of files describing all dependencies, resources, and parameters required for deploying an application within a K8s cluster. Objective: The goal of this study is to mine and empirically evaluate the security of Helm charts, comparing the performance of existing tools in terms of misconfigurations reported by policies available by default, and measure to what extent LLMs could be used for removing misconfiguration. We also want to investigate whether there are false positives in both the LLM refactorings and the tool outputs. Method: We propose a pipeline to mine Helm charts from Artifact Hub, a popular centralized repository, and analyze them using state-of-the-art open-source tools, such as Checkov and KICS. First, such a pipeline will run several chart analyzers and identify the common and unique misconfigurations reported by each tool. Secondly, it will use LLMs to suggest mitigation for each misconfiguration. Finally, the chart refactoring previously generated will be analyzed again by the same tools to see whether it satisfies the tool's policies. At the same time, we will also perform a manual analysis on a subset of charts to evaluate whether there are false positive misconfigurations from the tool's reporting and in the LLM refactoring.