REPOFUSE: Repository-Level Code Completion with Fused Dual Context

TL;DR

REPOFUSE introduces a novel repository-level code completion method that fuses analogy and rationale contexts using RTG to improve accuracy and speed without increasing latency.

Contribution

It presents a new fusion approach and RTG technique to enhance code completion efficiency and accuracy at the repository level.

Findings

Achieved up to 59.75% increase in exact match accuracy.

Improved inference speed by 26.8%.

Successfully integrated into enterprise workflows.

Abstract

The success of language models in code assistance has spurred the proposal of repository-level code completion as a means to enhance prediction accuracy, utilizing the context from the entire codebase. However, this amplified context can inadvertently increase inference latency, potentially undermining the developer experience and deterring tool adoption - a challenge we termed the Context-Latency Conundrum. This paper introduces REPOFUSE, a pioneering solution designed to enhance repository-level code completion without the latency trade-off. REPOFUSE uniquely fuses two types of context: the analogy context, rooted in code analogies, and the rationale context, which encompasses in-depth semantic relationships. We propose a novel rank truncated generation (RTG) technique that efficiently condenses these contexts into prompts with restricted size. This enables REPOFUSE to deliver precise code completions while maintaining inference efficiency. Through testing with the CrossCodeEval suite, REPOFUSE has demonstrated a significant leap over existing models, achieving a 40.90% to 59.75% increase in exact match (EM) accuracy for code completions and a 26.8% enhancement in inference speed. Beyond experimental validation, REPOFUSE has been integrated into the workflow of a large enterprise, where it actively supports various coding tasks.