Heimdallr: Characterizing and Detecting LLM-Induced Security Risks in GitHub CI Workflows

TL;DR

This paper studies security risks introduced by integrating Large Language Models into GitHub CI workflows, characterizes potential threats, and presents Heimdallr, a framework for detecting such risks with high accuracy.

Contribution

It provides the first comprehensive analysis of LLM-induced security risks in CI workflows and introduces Heimdallr, a novel hybrid analysis framework for risk detection.

Findings

Heimdallr achieves F1~=~0.994 in LLM-node identification.

Triggerability classification accuracy is 99.8%.

Detected and disclosed 802 vulnerable workflow instances.

Abstract

GitHub Continuous Integration (CI) workflows increasingly integrate Large Language Models (LLMs) to automate review, triage, content generation, and repository maintenance. This creates a new attack surface: externally controllable workflow inputs can shape LLM prompts and outputs, which may in turn affect security decisions, repository state, or privileged execution. Although LLM security and CI security have each been studied extensively, their intersection remains underexplored. In this paper, we present the first study of LLM-induced security risks in GitHub CI workflows. We characterize the problem along the full execution chain and develop a taxonomy of high-level risk classes and concrete threat vectors. To detect such risks in practice, we design Heimdallr, a hybrid analysis framework that normalizes workflows into an LLM-Workflow Property Graph (L-WPG) and combines triggerability analysis, LLM-assisted dataflow summarization, and deterministic propagation to synthesize concrete threat-vector findings. Evaluated on 300 manually annotated unique workflows, Heimdallr achieves high accuracy on LLM-node identification (F1~=~0.994), triggerability classification (99.8%), and threat-vector detection (micro-average F1~=~0.917). As part of an ongoing detection and disclosure effort, we have so far responsibly disclosed 802 vulnerable workflow instances across 759 repositories and received 71 acknowledgments.