Supporting Students in Navigating LLM-Generated Insecure Code

TL;DR

This paper introduces Bifröst, an educational framework that enhances students' ability to identify and analyze insecure code generated by large language models in AI-assisted software development.

Contribution

It presents a novel integrated educational tool combining a VS Code extension, adversarial LLMs, and feedback to improve security awareness in AI-augmented coding.

Findings

Students showed increased skepticism towards LLM-generated code after using Bifröst.

Classroom deployment revealed students' vulnerability to insecure code.

Post-intervention survey indicated improved security evaluation skills.

Abstract

The advent of Artificial Intelligence (AI), particularly large language models (LLMs), has revolutionized software development by enabling developers to specify tasks in natural language and receive corresponding code, boosting productivity. However, this shift also introduces security risks, as LLMs may generate insecure code that can be exploited by adversaries. Current educational approaches emphasize efficiency while overlooking these risks, leaving students underprepared to identify and mitigate security issues in AI-assisted workflows. To address this gap, we present Bifr\"ost, an educational framework that cultivates security awareness in AI-augmented development. Bifr\"ost integrates (1) a Visual Studio Code extension simulating realistic environments, (2) adversarially configured LLMs that generate insecure code, and (3) a feedback system highlighting vulnerabilities. By immersing students in tasks with compromised LLMs and providing targeted security analysis, Bifr\"ost cultivates critical evaluation skills; classroom deployments (n=61) show vulnerability to insecure code, while a post-intervention survey (n=21) indicates increased skepticism toward LLM outputs.