Your code assistant serves inline completions and chat from the same mid-size model cluster. Completion p95 is 900 ms end to end, and acceptance rate drops sharply above about 400 ms. Chat users are happy with multi-second responses. What is the best architectural change for completions?

1. A Keep the mid-size model but stream ghost text token by token, since users perceive the suggestion as instant once the first characters render.
2. B Route completions to a dedicated small code model pool with prefix caching of file context, accepting weaker multi-line suggestions.
3. C Expand context to the whole repository so suggestions are accepted more often, amortizing the latency cost across fewer total requests.
4. D Keep one shared large model for chat and completion, but give completion requests strict priority in the batch scheduler to remove queueing.