Your completion prompt is assembled as [snippets from other files, re-sorted by relevance on every keystroke] + [current file up to the cursor] + [suffix]. Prefix-cache hit rate on the inference servers is 8%, and prefill dominates latency. What is the cause and the fix?

1. A Cache keys include the code after the cursor; hash only the prefix so edits below the cursor stop invalidating cached entries.
2. B KV cache eviction is too aggressive; reserve more GPU memory for cached prefixes so they survive longer between keystrokes.
3. C Snippet retrieval is too slow; precompute embeddings for every file so retrieved context is ready before the request is sent.
4. D Reordered snippets change the leading tokens; put stable content first and keep snippet order fixed across nearby keystrokes.