A Lightweight Framework for Adaptive Retrieval In Code Completion With Critique Model

TL;DR

This paper introduces CARD, a lightweight critique framework that enhances retrieval efficiency and accuracy in code completion systems by intelligently selecting necessary retrievals, reducing latency and computational costs.

Contribution

CARD is a novel, generalizable critique method that seamlessly integrates into RAG-based code completion systems to improve efficiency and accuracy with minimal training and inference time.

Findings

Reduces retrievals by up to 46%, saving time and resources.

Improves code completion accuracy across multiple tasks.

Significantly decreases latency in code prediction tasks.

Abstract

Recent advancements in Retrieval-Augmented Generation have significantly enhanced code completion at the repository level. Various RAG-based code completion systems are proposed based on different design choices. For instance, gaining more effectiveness at the cost of repeating the retrieval-generation process multiple times. However, the indiscriminate use of retrieval in current methods reveals issues in both efficiency and effectiveness, as a considerable portion of retrievals are unnecessary and may introduce unhelpful or even harmful suggestions to code language models. To address these challenges, we introduce CARD, a lightweight critique method designed to provide insights into the necessity of retrievals and select the optimal answer from multiple predictions. CARD can seamlessly integrate into any RAG-based code completion system. Our evaluation shows that CARD saves 21% to 46% times of retrieval for Line completion, 14% to 40% times of retrieval for API completion, and 6% to 46.5% times of retrieval for function completion respectively, while improving the accuracy. CARD reduces latency ranging from 16% to 83%. CARD is generalizable to different LMs, retrievers, and programming languages. It is lightweight with training in few seconds and inference in few milliseconds.