Monday, August 23, 2010

"More Bandwidth" Does Not Necessarily Cure Latency Issues

Though it can help, increasing bandwidth on any network experiencing latency issues does not necessarily fix the problem.

Congestion and forwarding delay are the more important types of latency on an network, and are not entirely independent. As a network element is subjected to heavy load, it may need additional queue time to handle and
process the increased volume of traffic, which causes forwarding delay.

But there are other sources, as well. Serialization delay is the most constant, having only a small influence on end-to-end latency. Propagation delay, typically stable in circuit-switched networks, can be irregular and introduce jitter over routed networks.

As network congestion can have a large impact on end-to-end latency, affecting both forwarding and pure congestion (queuing-related) delay.

Reducing traffic bottlenecks therefore is a key part of network management and design. Increasing capacity (available bandwidth) should, at least in theory, help reduce congestion when applied to network “pinch points”.

However, increasing throughput does not always lead to the expected decrease in latency, even if
congestion is reduced. Results will vary depending on implementation, network architecture, traffic
patterns, and a number of other factors.

white paper here

No comments:

"Tokens" are the New "FLOPS," "MIPS" or "Gbps"

Modern computing has some virtually-universal reference metrics. For Gemini 1.5 and other large language models, tokens are a basic measure...