What a Load Balancer Does
A load balancer sits between your users and your servers. Its job is simple: send each request to one of many servers, so no single server gets too much. It also notices when a server is sick and stops sending traffic to it.
Layer 4 vs Layer 7
Load balancers operate at different layers of the network stack, and this distinction matters more than most people realize.
Routes based on IP address and TCP/UDP port. It does not look inside the request. It only forwards the connection.
- Faster (no payload inspection)
- Protocol-agnostic
- Cannot route by URL, headers, or cookies
- Example: AWS NLB, HAProxy in TCP mode
Looks inside the HTTP request: URL path, headers, cookies, body. Can make intelligent routing decisions.
- Route
/api/*to API servers - Route
/static/*to CDN origin - Sticky sessions via cookies
- Example: AWS ALB, Nginx, Envoy
Use L4 when you need raw throughput and do not care about request content, database connections, non-HTTP protocols, or extremely high-volume TCP traffic. Use L7 when you need content-aware routing, which is most web applications.
Routing Algorithms
| Algorithm | How it works | Best for |
|---|---|---|
| Round Robin | Cycles through servers sequentially | Identical servers, uniform requests |
| Weighted Round Robin | Like round robin, but bigger servers get more requests | Mixed server sizes |
| Least Connections | Sends to the server with fewest active connections | Requests with varying processing time |
| IP Hash | Hashes client IP to consistently map to the same server | Session affinity without cookies |
| Random | Picks a server at random | Statistically equivalent to round robin at scale |
Reverse Proxy
A reverse proxy sits in front of your servers and forwards client requests to them. "But that is what a load balancer does?" Yes. A load balancer is a type of reverse proxy. But reverse proxies do more:
- SSL termination: Handle TLS encryption/decryption so your backend servers do not have to.
- Compression: Gzip/Brotli compress responses before sending to clients.
- Caching: Serve cached responses for repeat requests without hitting the backend.
- Security: Hide your server topology from clients. They only see the proxy's IP.
- Rate limiting: Throttle abusive clients before they reach your application.
Even if you only have one backend server, a reverse proxy adds value. SSL termination alone is worth it. It keeps certificate management in one place and lets your app server focus on application logic.
Health Checks
A load balancer is only useful if it knows which servers are healthy. Health checks are periodic probes sent to each server:
- Active health checks: The LB pings each server (e.g.,
GET /health) every few seconds. If a server fails N consecutive checks, it is removed from rotation. - Passive health checks: The LB monitors real traffic. If a server starts returning 5xx errors or timing out, it is marked unhealthy.
Watch what the user sees when one server stops. Move the health check interval. A short interval finds the dead server faster. A retry hides the gap from the user.
Load balancing · Health checks
What the user sees when one server stops
Health checks and retries decide what the user sees when a server dies. The check interval decides how long it hurts.
The scene sends 2 requests each second and checks health on a fixed timer. Real load balancers also mark a server bad after a few failed real requests (passive checks). A short check interval finds a dead server faster but adds load. Most teams pick 2 to 10 seconds and 2 to 3 failed checks in a row.