FREE LESSON · Networks & distributed systems · 2 OF 4
Retries, Timeouts and Exponential Backoff
Failure is partial and ambiguous — Timeouts, retries, backoff, and overload
A timeout bounds waiting; it does not identify what happened.
The request, server work, or response may have been lost or delayed. Retrying can recover transient failure but may duplicate effects and amplify overload. Deadlines propagate a total time budget; exponential backoff with jitter spreads retry traffic; circuit breakers and admission control protect a struggling dependency.
Every retry is additional load placed on a system already suspected of trouble.
Little’s law reveals hidden queues
For a stable system, average concurrency is approximately throughput multiplied by average time in system. At 1,000 requests per second and 200 ms average latency, about 200 requests are in flight. If service capacity falls while arrivals continue, the queue grows and latency rises before outright errors appear.
Overload often looks like slowness first; a bounded queue makes the failure visible and recoverable.