One pattern I increasingly observe in system engineering is the disproportionate emphasis on client-to-server and server-to-client communication latency, often without adequately distinguishing network transit time from application processing time.
Regardless of the communication protocol, caching strategy, CDN, memoisation, or processing-layer optimisation, every distributed system operates within a fundamental physical constraint: data takes time to travel.
You can optimise computation. You can reduce payload sizes, reuse connections, eliminate redundant database queries, introduce caching, and improve network routing. But you cannot optimise away the physical distance between two endpoints.
This becomes particularly interesting when a system serves users within a geographically constrained demographic boundary, such as India.
1. Network latency has a mathematical lower bound
India has a relatively small number of major cloud infrastructure regions compared with its geographical spread.
Consider the speed of light in optical fibre, approximately 200,000 km/s, or roughly two-thirds of its speed in a vacuum.
This gives us a useful theoretical approximation:
One-way propagation latency: approximately 5 microseconds per kilometre.
Round-trip propagation latency (RTT): approximately 10 microseconds per kilometre.
Theoretical minimum round-trip latency (RTT):
500 km → 5 ms
1,000 km → 10 ms
1,500 km → 15 ms
2,000 km → 20 ms
Note: These are theoretical minimums based on light travelling through optical fibre, not real-world network latency.
These numbers represent ideal propagation limits, not achievable end-to-end network benchmarks. Actual fibre routes are rarely geographically direct. Routers, ISP peering, congestion, last-mile connectivity, packet processing and retransmissions introduce additional latency.
Nevertheless, this gives engineering teams a useful starting point for establishing realistic network budgets.
The important insight is that geographical distance is not merely an infrastructure consideration. It is part of the system's latency equation.
2. Establish a latency budget before optimising
One of the most important distinctions in performance engineering is separating network transit latency from application processing latency.
Consider a request where:
Network communication contributes 80 ms.
Reverse proxy and application processing contribute 20 ms.
Database operations contribute 20 ms.
The resulting request latency is approximately 120 ms, assuming the components execute sequentially and there are no additional network round trips.
Now imagine the engineering team spends two weeks optimising Node.js processing and reduces application execution from 20 ms to 5 ms.
The total becomes approximately 105 ms.
That is an improvement, but the maximum possible gain from that particular optimisation was only 15 ms.
If the network contributes 80 ms, optimising application code indefinitely cannot eliminate that dominant component.
Conversely, if application processing consumes 100 ms while network transit contributes only 10 ms, the engineering effort should focus on computation, database access, contention, serialization or architectural bottlenecks.
Optimising the wrong latency component is one of the most expensive mistakes in performance engineering.
3. Treat latency as an explicit engineering budget
For systems serving users across India through a centrally hosted application, I would initially propose the following network RTT objectives:
Under 20 ms: excellent regional connectivity.
20–35 ms: a strong target across major metropolitan routes.
35–50 ms: acceptable for many intercity routes.
50–80 ms: a reasonable upper operational budget for more geographically distant users.
Above 80 ms: investigate routing, ISP connectivity, congestion and regional infrastructure placement.
These are proposed engineering targets, not cloud-provider guarantees. They should be validated using measurements from representative user networks and locations.
For synchronous application processing, I would initially aim for sub-100 ms where practical, with 200 ms as a proposed upper budget for workloads where the interaction permits it.
The actual budget must depend on the user experience, workload, request complexity and service-level objectives.
More importantly, measure p50, p95 and p99. Average latency alone can conceal serious tail-latency problems.
4. Know when synchronous execution stops making sense
There is another architectural decision that follows naturally from latency budgeting.
Suppose a request requires:
Multiple database operations.
Third-party API calls.
Blockchain transactions.
Document generation.
Notifications.
Several independent downstream services.
If the combined execution time consistently exceeds the synchronous latency budget, continuing to optimise the same request path may be the wrong architectural decision.
This is where asynchronous execution becomes valuable.
Instead of holding the client connection while all operations complete:
Validate the request.
Persist or durably enqueue the work.
Return an acknowledgement with a request or job identifier.
Execute the workload asynchronously.
Notify the client or expose a status endpoint when the result is ready.
This changes the interaction model from waiting for completion to acknowledging accepted work.
However, an acknowledgement must only be returned after the work has been durably accepted. Otherwise, the system risks acknowledging operations that are subsequently lost.
5. A practical benchmarking model
I would encourage engineering teams to decompose request latency into independently observable components:
End-to-end latency = client/network transit + server-side processing + downstream dependency latency + response transit.
Within server-side processing, further separate:
Reverse proxy and load-balancer overhead.
Application execution.
Queueing and event-loop delays.
Database execution.
Internal service-to-service communication.
External dependency calls.
Instrument these separately using distributed tracing and request correlation IDs.
One important measurement detail: server processing time should be measured using server-side spans, while client-observed latency should be measured at the client. Subtracting server execution time from client-observed time only gives an approximate residual unless clock synchronisation and measurement boundaries are handled correctly.
The larger engineering principle
A scalable system is not necessarily one that makes every operation faster. It is one that understands where time is consumed, establishes realistic latency budgets, and makes deliberate architectural decisions around those constraints.
Caching, CDN, protocol selection, connection pooling, database optimisation, horizontal scaling and asynchronous processing are all useful tools. But their effectiveness depends on identifying the dominant latency component first.
Before optimising a system, establish where its milliseconds are going. Then spend engineering effort where it can produce the greatest measurable reduction in end-to-end latency.