FREE Systems LESSON · Systems
Follow the work, including the waiting
A request across machine boundaries
Latency is a path plus its queues.
A request consumes CPU only intermittently. It may wait for a run queue, a cache line, a page fault, a socket, a connection pool, a downstream service, a lock, or durable storage. Average component times cannot explain tail latency when queues interact. Trace the critical path, attach an owner and budget to each boundary, and distinguish service time from waiting time.
The slowest request is often not doing more work; it is arriving at the wrong queue.
A timeout is a budget, not a cure.
Increasing timeouts can convert a visible failure into a larger queue of doomed work. Retries can multiply load precisely when the dependency is least able to serve it. Deadlines must propagate; concurrency must be bounded; retries need budgets, jitter, and idempotency. Otherwise a local resilience feature becomes a system-wide amplifier.
Overload control decides which work the system will refuse so the rest can finish.