Communication

HTTP, TCP, UDP, RPC, and REST: how your services talk to each other, and which one to pick.

The Protocol Stack

Every message on a network goes through a stack of protocols. When you know which layer you are working at, you make better choices.

Application (HTTP, gRPC, WebSocket)
Transport (TCP, UDP)
Network (IP)
Link (Ethernet, Wi-Fi)
Each layer builds on the one below. Higher layers trade raw speed for richer semantics.

TCP vs UDP

TCP (Transmission Control Protocol)

Connection-oriented. Guarantees delivery, ordering, and error checking. Slower due to handshake and acknowledgments.

  • 3-way handshake to establish connection
  • Retransmits lost packets
  • Flow control and congestion avoidance
  • Use for: HTTP, databases, file transfer, email
UDP (User Datagram Protocol)

Connectionless. Fire-and-forget. No delivery guarantee, no ordering, no handshake. Fast.

  • No connection setup overhead
  • Lost packets are just lost
  • Lower latency, higher throughput
  • Use for: Video streaming, gaming, DNS, VoIP

The rule of thumb: Use TCP when data integrity matters (you cannot lose or reorder packets). Use UDP when speed matters more than completeness (a dropped video frame is fine, but a lagged stream is not).

HTTP

HTTP is the application-layer protocol that powers the web. It is request-response based: a client sends a request with a method, URL, headers, and optional body. The server returns a status code, headers, and optional body.

HTTP Methods

Method Purpose Idempotent? Has body?
GET Read a resource Yes No
POST Create a resource or trigger action No Yes
PUT Replace a resource entirely Yes Yes
PATCH Partially update a resource No* Yes
DELETE Remove a resource Yes Optional
Idempotency is crucial for reliability. An idempotent operation produces the same result whether you call it once or ten times. This matters because networks fail. If a request times out, the client retries, and you do not want double-charges or duplicate records. Design your APIs so that retries are safe.

REST

REST (Representational State Transfer) is an architectural style for APIs, not a protocol. A RESTful API models your domain as resources with standard CRUD operations mapped to HTTP methods:

Operation HTTP URL Example
List GET /users Get all users
Read GET /users/42 Get user 42
Create POST /users Create new user
Update PUT /users/42 Replace user 42
Delete DELETE /users/42 Delete user 42

REST works well when your domain maps naturally to resources and CRUD operations. It is intuitive, cacheable (GET responses are inherently cacheable), and widely tooled.

REST Limitations

  • Over-fetching: GET /users/42 returns the entire user object even if you only need the name.
  • Under-fetching: Getting a user's profile, posts, and followers requires 3 separate API calls.
  • Action-oriented operations: "Send a password reset email" does not map cleanly to a resource. You end up with awkward endpoints like POST /password-reset-requests.

RPC

RPC (Remote Procedure Call) treats remote service calls like local function calls. Instead of "GET the user resource at this URL," it is "call the getUser function on the remote server with these arguments."

REST Style
GET /users/42/orders?status=active

Resource-oriented. Standard HTTP semantics.

RPC Style
getActiveOrders(userId: 42)

Action-oriented. Explicit function call.

Modern RPC frameworks like gRPC use Protocol Buffers for serialization (much smaller and faster than JSON) and HTTP/2 for transport (multiplexing, streaming, header compression). This makes them significantly more efficient than REST for service-to-service communication.

Aspect REST RPC (gRPC)
Model Resources (nouns) Functions (verbs)
Serialization JSON (text) Protobuf (binary)
Streaming Limited (SSE, WebSocket) Native bidirectional
Browser support Native Requires grpc-web proxy
Best for Public APIs, web clients Internal microservices
A common pattern: REST for your public API (client-facing, browser-friendly) and gRPC for internal service-to-service communication (fast, strongly typed, streaming support). You get the best of both worlds.

Streaming a Response

Some answers take a long time to make. A chat model writes one word at a time. The server can wait and send the full answer at the end. Or it can send each word as soon as it is ready, with server-sent events or a WebSocket. The total time is the same. The wait feels different. Guess first, then press Play.

AI Systems · LLM inference serving

Streaming against a buffered answer

Streaming does not make the answer faster. It makes the wait feel shorter. It also changes what the user gets when something breaks.

Predict first

Two servers write the same 100-word answer at the same speed. One sends it word by word. One sends it all at the end. Which one finishes first?

They finish at the same time

Both servers write words at the same speed. Streaming changes when you see the first word, not when the last word is written.

Buffered

One response at the end
Ready
A website stays up when one server stops because a load balancer sits in front of the servers. The load balancer gets every request first. It sends each request to a server that is healthy. It asks each server every few seconds if it is ok. When a server does not answer, the load balancer takes it out of the list. New requests go to the servers that are left. The user may see one or two slow requests. Then everything works again. This is why big sites run more than one server for each job. One server can stop at any time. The site keeps going because the others pick up the work. A retry helps too. If a request fails, the load balancer can send it again to another server. The user does not see the error. The total time is a little longer. The answer still arrives.
first word – done –

Streaming

Each word as it is written
Ready
A website stays up when one server stops because a load balancer sits in front of the servers. The load balancer gets every request first. It sends each request to a server that is healthy. It asks each server every few seconds if it is ok. When a server does not answer, the load balancer takes it out of the list. New requests go to the servers that are left. The user may see one or two slow requests. Then everything works again. This is why big sites run more than one server for each job. One server can stop at any time. The site keeps going because the others pick up the work. A retry helps too. If a request fails, the load balancer can send it again to another server. The user does not see the error. The total time is a little longer. The answer still arrives.
first word – done –

The scene counts words as tokens. A real token is about three quarters of a word. First token time is fixed at 0.4 s to stand for queue time plus prefill. Streaming uses server-sent events or a WebSocket. A buffered answer is one HTTP response.

Streaming changes the error model. A buffered response fails as a whole. A streamed response can stop in the middle. The client must know the difference between "the answer ended" and "the connection broke". Send a clear end marker, and show the user when the answer was cut off.

GraphQL

GraphQL solves the over-fetching and under-fetching problems of REST. The client specifies exactly what data it needs in a single query, and the server returns exactly that: nothing more, nothing less.

Trade-off: GraphQL shifts complexity to the server (resolvers, query optimization, N+1 prevention) and eliminates HTTP-level caching (everything is a POST). It shines for complex, nested data requirements (mobile apps, dashboards) but is overkill for simple CRUD APIs.