SecCoder: Towards Generalizable and Robust Secure Code Generation

TL;DR

SecCoder introduces a novel secure code generation method using in-context learning and safe demonstrations, significantly improving security, generalizability, and robustness against attacks in code generation tasks.

Contribution

It proposes a new secure code generation approach that enhances security and robustness using in-context learning and a dense retriever for demonstration selection.

Findings

Achieves 7.20% security improvement on unseen test cases.

Achieves 7.74% security improvement against attacked models.

Demonstrates superior generalizability and robustness over existing methods.

Abstract

After large models (LMs) have gained widespread acceptance in code-related tasks, their superior generative capacity has greatly promoted the application of the code LM. Nevertheless, the security of the generated code has raised attention to its potential damage. Existing secure code generation methods have limited generalizability to unseen test cases and poor robustness against the attacked model, leading to safety failures in code generation. In this paper, we propose a generalizable and robust secure code generation method SecCoder by using in-context learning (ICL) and the safe demonstration. The dense retriever is also used to select the most helpful demonstration to maximize the improvement of the generated code's security. Experimental results show the superior generalizability of the proposed model SecCoder compared to the current secure code generation method, achieving a significant security improvement of an average of 7.20% on unseen test cases. The results also show the better robustness of SecCoder compared to the current attacked code LM, achieving a significant security improvement of an average of 7.74%. Our analysis indicates that SecCoder enhances the security of LMs in generating code, and it is more generalizable and robust.