Optimizing LLM Code Suggestions: Feedback-Driven Timing with Lightweight State Bounds

TL;DR

This paper introduces an adaptive timing mechanism for LLM code suggestions that uses real-time feedback and developer state predictions to improve acceptance rates and reduce wasted inference calls.

Contribution

It presents a novel feedback-driven timing method that dynamically adjusts suggestion delays based on developer responses and cognitive state predictions.

Findings

Suggestion acceptance increased from 4.9% to 18.6%.

Wasted inference calls reduced by 75%.

Blind rejections decreased from 8.3% to 0.36%.

Abstract

Large Language Models (LLMs) have transformed code auto-completion by generating context-aware suggestions. Yet, deciding when to present these suggestions remains underexplored, often leading to interruptions or wasted inference calls. We propose an adaptive timing mechanism that dynamically adjusts the delay before offering a suggestion based on real-time developer feedback. Our suggested method combines a logistic transform of recent acceptance rates with a bounded delay range, anchored by a high-level binary prediction of the developer's cognitive state. In a two-month deployment with professional developers, our system improved suggestion acceptance from 4.9% with no delay to 15.4% with static delays, and to 18.6% with adaptive timing-while reducing blind rejections (rejections without being read) from 8.3% to 0.36%. Together, these improvements increase acceptance and substantially reduce wasted inference calls by 75%, making LLM-based code assistants more efficient and cost-effective in practice.