Large Language Models for Code Generation: A Comprehensive Survey of Challenges, Techniques, Evaluation, and Applications

TL;DR

This survey reviews the capabilities, challenges, techniques, evaluation metrics, and applications of Large Language Models in automated code generation, highlighting their current state and future potential.

Contribution

It provides a comprehensive overview of LLMs for code generation, including recent techniques, evaluation methods, and practical applications, serving as a resource for researchers.

Findings

LLMs can effectively generate executable code from natural language.

Fine-tuning techniques improve LLM performance and adaptability.

Applications like GitHub Copilot demonstrate practical utility.

Abstract

Large Language Models (LLMs) have demonstrated their remarkable capabilities in numerous fields. This survey focuses on how LLMs empower users, regardless of their technical background, to use human languages to automatically generate executable code. We begin with understanding LLMs' limitations and challenges in automated code generation. Subsequently, we review various fine-tuning techniques designed to enhance both the performance and adaptability of LLMs in code generation tasks. We then review the existing metrics and benchmarks for evaluations to assess model performance based on fine-tuning techniques. Finally, we explore the applications of LLMs (e.g. CodeLlama, GitHub Copilot, ToolGen) in code generation tasks to illustrate their roles and functionalities. This survey provides a comprehensive overview of LLMs for code generation, helps researchers in diverse fields better understand the current state-of-the-art technologies, and offers the potential of effectively leveraging LLMs for code generation tasks.