Specification and Detection of LLM Code Smells

TL;DR

This paper introduces the concept of LLM code smells, formalizes five problematic coding practices, and extends a detection tool to identify their prevalence in open-source LLM systems, impacting over half of the analyzed projects.

Contribution

It defines and formalizes LLM-specific code smells and extends a detection tool, providing the first systematic approach to identify and analyze these issues in software systems.

Findings

LLM code smells affect 60.50% of analyzed systems.

Detection precision of the extended tool is 86.06%.

Five recurrent problematic coding practices related to LLM inference are identified.

Abstract

Large Language Models (LLMs) have gained massive popularity in recent years and are increasingly integrated into software systems for diverse purposes. However, poorly integrating them in source code may undermine software system quality. Yet, to our knowledge, there is no formal catalog of code smells specific to coding practices for LLM inference. In this paper, we introduce the concept of LLM code smells and formalize five recurrent problematic coding practices related to LLM inference in software systems, based on relevant literature. We extend the detection tool SpecDetect4AI to cover the newly defined LLM code smells and use it to validate their prevalence in a dataset of 200 open-source LLM systems. Our results show that LLM code smells affect 60.50% of the analyzed systems, with a detection precision of 86.06%.