Latency is a product decision

Teams screenshot a p50 and call it performance. Users live in the tail. A trading desk, a checkout, a live session — they feel P99. If that number is a rumour, the product is a rumour.

We have moved critical paths off a Node hop that sat above 100ms and onto a Go/gRPC fabric that holds around ten thousand TPS with P99 under 25ms. That sentence is not a flex. It is a list of choices: where state lives, what is allowed to block, and which fan-out is allowed to exist.

What actually moves the tail

Hot order state in Redis, not a round-trip to the system of record on every tick. Idempotent handlers so retries do not double-apply. Goroutine pools for connector fan-out instead of unbounded hope. An audit log that is append-only, so the hot path is not also the historian.

You cannot sprinkle this on later. If the first draft of the service talks JSON over a sleepy hop, the P99 is already decided. The product decision happened in week one, dressed as “we will optimise.”

How to talk about it

Do not lead with a library. Lead with the SLA the business already promised. Then draw the path a single order takes, and delete every hop that is not load-bearing. The remaining diagram is the architecture. Everything else is a meeting.