LLMs Meet Library Evolution: Evaluating Deprecated API Usage in LLM-based Code Completion
Chong Wang, Kaifeng Huang, Jian Zhang, Yebo Feng, Lyuye Zhang, Yang, Liu, and Xin Peng

TL;DR
This paper evaluates how large language models handle deprecated API usage in code completion, revealing current challenges and proposing baseline solutions to improve API update handling in evolving libraries.
Contribution
It is the first comprehensive evaluation of deprecated API usage in LLM-based code completion and introduces two baseline fixing approaches, REPLACEAPI and INSERTPROMPT.
Findings
High deprecated API usage rate in LLM outputs
Model and prompt influence deprecated API usage
Baseline approaches reduce deprecated API usage
Abstract
Large language models (LLMs), pre-trained or fine-tuned on large code corpora, have shown effectiveness in generating code completions. However, in LLM-based code completion, LLMs may struggle to use correct and up-to-date Application Programming Interfaces (APIs) due to the rapid and continuous evolution of libraries. While existing studies have highlighted issues with predicting incorrect APIs, the specific problem of deprecated API usage in LLM-based code completion has not been thoroughly investigated. To address this gap, we conducted the first evaluation study on deprecated API usage in LLM-based code completion. This study involved seven advanced LLMs, 145 API mappings from eight popular Python libraries, and 28,125 completion prompts. The study results reveal the status quo (i.e., API usage plausibility and deprecated usage rate) of deprecated API and replacing API usage in…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDigital Rights Management and Security · Service-Oriented Architecture and Web Services
