Nginx, Caddy, and Traefik: Why Connection Churn Splits Latency

Archi4 min read

nginxtlsnetworkingproxies

Contents
  1. Three Proxies, One Workload, Three Bottlenecks
  2. Nginx: worker-local state meets a shared listen socket
  3. Caddy: goroutines, garbage collection, and automatic TLS
  4. Traefik: the routing table is the hot path
  5. Which root cause fits your graph

Three Proxies, One Workload, Three Bottlenecks#

A reverse proxy comparison usually turns into a feature table. That table hides the reason p99 latency diverges when clients open and close TLS connections quickly. Nginx, Caddy, and Traefik all terminate TLS and forward HTTP. The difference sits lower: how each one distributes accepted sockets, stores session state, and applies configuration changes.

Use a concrete workload. A 4 vCPU VM, 2,000 idle keepalive connections, 400 new TLS handshakes per second, and a backend that answers in 8 ms. CPU sits near 40 percent. Nginx may show a flat median and a p99 spike every 30 seconds. Caddy may show a smoother p99 but a memory sawtooth. Traefik may track both until a service deploy triggers a route rebuild. Same traffic, different root causes.

Nginx: worker-local state meets a shared listen socket#

Nginx uses a fixed number of worker processes. Each worker runs an event loop, accepts connections, and owns its upstream keepalive pool. The kernel distributes new connections across workers. Under connection churn, that design creates two costs.

First, upstream connection reuse is per worker. If you set 32 keepalive connections to the backend, four workers can hold up to 128. That is not inherently bad. It becomes visible when one worker gets a burst of slow requests and its pool empties while other workers sit with idle backend sockets. The backend sees uneven load.

Second, TLS session state is not automatically global. If ssl_session_cache is not configured as shared, or ticket keys are not aligned across reloads, resumed handshakes can miss. A client that reconnects every few hundred milliseconds pays for a full handshake more often than the configuration suggests. The CPU cost lands inside the event loop, where it delays every connection on that worker.

Reloads add another layer. Nginx starts new workers and drains old ones. With long keepalives and rapid deploys, old workers can linger for minutes. Their backend pools remain open. A monitoring graph shows four workers when you expect two, and the extra processes explain a slice of the latency.

Caddy: goroutines, garbage collection, and automatic TLS#

Caddy handles connections through Go's runtime. Each connection gets a goroutine. The scheduler maps many goroutines onto OS threads, so there is no fixed worker count and no per-worker upstream pool with the same shape as Nginx. That removes one source of imbalance.

It introduces another. TLS handshakes allocate. A new connection creates buffers, certificate state, and HTTP parser structures. Under 400 handshakes per second, the Go garbage collector has more work. The pauses are short, often sub-millisecond, but they hit all goroutines in the process. A p99 graph can show a regular ripple that lines up with GC cycles, not with backend time.

Caddy's automatic HTTPS is convenient and technically relevant. Certificate renewal, OCSP stapling, and session ticket rotation happen inside the same process. When a ticket key rotates, a client with an old ticket must do a full handshake. That is correct behavior. It also means a proxy that looks idle can show a handshake burst every rotation interval.

Caddy's admin API reloads configuration without a hard restart. That helps with old-worker drain. It does not make handshakes free. If the p99 spike follows a certificate or ticket event, the root cause is TLS state, not routing.

Traefik: the routing table is the hot path#

Traefik also runs on Go, so goroutine scheduling and GC apply. Its distinct cost is dynamic configuration. Traefik watches providers such as Docker, Kubernetes, or Consul, then builds routers and middleware chains. Every request passes through route matching and the middleware stack.

With a few dozen routers, that work is noise. With several thousand, route matching and per-request allocations become measurable. The symptom looks like proxy latency, but the backend sees normal response times. The delay sits between the TLS terminator and the upstream dial.

Provider churn makes it worse. A deploy that adds or removes services triggers a configuration rebuild. If rebuilds happen while traffic is high, requests can wait on internal locks or run against a partly swapped routing table. A p99 spike that follows every deploy is a signal to inspect provider events, not just CPU.

Which root cause fits your graph#

Match the symptom to the mechanism. If p99 spikes correlate with connection count and CPU is moderate, check TLS session reuse and accept distribution first. For Nginx, confirm shared session cache and ticket key files across reloads. For Caddy, check whether the spike follows GC cycles or certificate events. For Traefik, compare deploy timestamps with latency.

Then check upstream pooling. Nginx's worker-local pools can strand idle backend sockets. Caddy and Traefik share process-wide state, so the imbalance is smaller, but their per-request allocations can still raise tail latency.

The practical test is a short connection load with a fixed backend. Run it for 10 minutes, then overlay TLS handshake counts, GC pauses, and provider event logs. The proxy with the best feature list may not be the one whose tail latency matches your traffic. The one that does is the one whose internal state model fits how your clients connect.

More articles