GitHub Considered Harmful? Analyzing Open-Source Projects for the Automatic Generation of Cryptographic API Call Sequences

TL;DR

This paper examines the quality of cryptographic API call sequences in GitHub code, highlighting prevalent misuses and proposing enhanced models to generate more secure cryptographic code from natural language descriptions.

Contribution

It provides a detailed analysis of cryptographic API call misuses in GitHub and introduces security-aware models to improve automatic code generation.

Findings

High prevalence of cryptographic API misuses in GitHub code

Augmented DeepAPI with security-specific training improves API call sequence accuracy

Neglecting API misuses leads to insecure code generation

Abstract

GitHub is a popular data repository for code examples. It is being continuously used to train several AI-based tools to automatically generate code. However, the effectiveness of such tools in correctly demonstrating the usage of cryptographic APIs has not been thoroughly assessed. In this paper, we investigate the extent and severity of misuses, specifically caused by incorrect cryptographic API call sequences in GitHub. We also analyze the suitability of GitHub data to train a learning-based model to generate correct cryptographic API call sequences. For this, we manually extracted and analyzed the call sequences from GitHub. Using this data, we augmented an existing learning-based model called DeepAPI to create two security-specific models that generate cryptographic API call sequences for a given natural language (NL) description. Our results indicate that it is imperative to not neglect the misuses in API call sequences while using data sources like GitHub, to train models that generate code.