The Current Challenges of Software Engineering in the Era of Large Language Models

TL;DR

This paper reviews the integration of large language models into software engineering, identifying 26 key challenges across the software development lifecycle to guide future research in this emerging field.

Contribution

It systematically summarizes the current landscape of LLMs in software engineering and highlights 26 challenges to address for advancing the field.

Findings

26 key challenges identified across SDLC stages

Challenges span requirements, coding, testing, review, maintenance, and security

Discussion involved over 20 experts from academia and industry

Abstract

With the advent of large language models (LLMs) in the artificial intelligence (AI) area, the field of software engineering (SE) has also witnessed a paradigm shift. These models, by leveraging the power of deep learning and massive amounts of data, have demonstrated an unprecedented capacity to understand, generate, and operate programming languages. They can assist developers in completing a broad spectrum of software development activities, encompassing software design, automated programming, and maintenance, which potentially reduces huge human efforts. Integrating LLMs within the SE landscape (LLM4SE) has become a burgeoning trend, necessitating exploring this emergent landscape's challenges and opportunities. The paper aims at revisiting the software development life cycle (SDLC) under LLMs, and highlighting challenges and opportunities of the new paradigm. The paper first summarizes the overall process of LLM4SE, and then elaborates on the current challenges based on a through discussion. The discussion was held among more than 20 participants from academia and industry, specializing in fields such as software engineering and artificial intelligence. Specifically, we achieve 26 key challenges from seven aspects, including software requirement & design, coding assistance, testing code generation, code review, code maintenance, software vulnerability management, and data, training, and evaluation. We hope the achieved challenges would benefit future research in the LLM4SE field.