Codexity: Secure AI-assisted Code Generation

TL;DR

Codexity is a security-focused framework that enhances AI-assisted code generation by integrating static analysis feedback to reduce vulnerabilities in generated software.

Contribution

It introduces a novel framework that combines LLMs with static analysis tools to improve the security of AI-generated code.

Findings

Prevents 60% of vulnerabilities in generated code.

Effective across five different LLMs.

Reduces security risks in AI-assisted software development.

Abstract

Despite the impressive performance of Large Language Models (LLMs) in software development activities, recent studies show the concern of introducing vulnerabilities into software codebase by AI programming assistants (e.g., Copilot, CodeWhisperer). In this work, we present Codexity, a security-focused code generation framework integrated with five LLMs. Codexity leverages the feedback of static analysis tools such as Infer and CppCheck to mitigate security vulnerabilities in LLM-generated programs. Our evaluation in a real-world benchmark with 751 automatically generated vulnerable subjects demonstrates Codexity can prevent 60% of the vulnerabilities being exposed to the software developer.