Latency, Bandwidth, Throughput, and the BDP

Build the mental model

Bandwidth, latency, and throughput get used interchangeably and mean three different things. Bandwidth is capacity: the maximum bits per second a link can carry. Latency is delay: how long one bit takes to get there. Throughput is what you actually achieve, and it is bounded by both plus everything else in the way.

Latency is not one number; it is a sum of four things. Propagation delay is distance divided by signal speed in the medium: physics, and nothing you buy reduces it. New York to London is roughly forty milliseconds one way and always will be. Transmission delay is packet size divided by link rate, the time to clock the bits onto the wire, and this is the only component more bandwidth improves. Queuing delay is time spent waiting in a router's buffer behind other traffic, and it is the most variable component, because it is what congestion actually feels like. Processing delay is the router's own lookup work, usually negligible today.

This decomposition explains the result people find most counterintuitive: upgrading a link from 100 Mbps to 1 Gbps often changes a transfer's completion time by almost nothing. If propagation dominates, you multiplied the small term and left the large one alone.

It gets sharper with the bandwidth-delay product. BDP is bandwidth times round-trip time, and it measures how many bytes are in flight, already sent but not yet acknowledged. A sender using a fixed window can never exceed window divided by RTT, no matter how wide the pipe is. On a 100 Mbps, 80 ms transatlantic link the BDP is nearly a megabyte, so a classic 64 KB window caps a single TCP flow at a few Mbps. Ten times the bandwidth changes that number not at all. The pipe got wider; the constraint was its length.

THE FOUR COMPONENTS OF LATENCY AND THE PIPE ANALOGY
---------------------------------------------------
ONE PACKET, START TO FINISH (not to scale)

sender                                                receiver
  |-proc-|--queuing---|--transmission--|--propagation--->|
     ^         ^              ^                ^
     |         |              |                |
  router    waiting in     size / link     distance / speed
  lookup    the buffer     rate            of signal
  (tiny)    (varies a lot) ^^^^^^^^^^^     (pure physics,
                           ONLY this one     you cannot
                           shrinks when      buy your way
                           you buy more      out of it)
                           bandwidth


PIPE ANALOGY

  bandwidth = how WIDE the pipe is
  latency   = how LONG the pipe is
  BDP       = how much water is INSIDE it right now

  narrow + short  |==|                   tiny BDP, 64KB is plenty
  narrow + long   |==================|   moderate BDP
  wide   + short  |####|                 tiny BDP
  wide   + long   |##################|   HUGE BDP, needs a big
                                         window or you idle

  Making the pipe wider does not make it shorter. A transfer
  limited by the LENGTH gains nothing from more WIDTH.

Connect it to a real scenario

You move a service to another region and file transfers that took twenty seconds now take four minutes, on a link the provider swears is 1 Gbps. Nobody believes the link. It is usually fine.

Do the arithmetic before you open a ticket. Measure RTT with ping, compute the bandwidth-delay product, and compare it against the TCP window in use. If the BDP is nine megabytes and the window is 64 KB, a single flow can never do better than about six Mbps, and it does not matter what the link is rated at, because the sender spends almost all its time waiting for acknowledgements rather than sending.

Two fixes follow directly. Enable window scaling and let the kernel auto-tune buffers, raising the ceiling for one flow. Or run several flows in parallel, which is exactly why multi-threaded download tools and object-storage clients feel so much faster on long links: each flow gets its own window, and the windows add up.

The same reasoning is what makes CDNs work. A CDN does not give you more bandwidth; it shortens the distance, which shrinks propagation delay, which shrinks the RTT that divides everything else. It also explains why chatty protocols hurt so badly over long links: every extra round trip costs a full RTT regardless of how few bytes it moved.

• Bandwidth is capacity, latency is delay, throughput is the result
• Only transmission delay improves when you buy more bandwidth
• BDP = bandwidth x RTT is the data in flight, and it sets the window you need

Try the working example

# Bandwidth is capacity (bits per second).
# Latency is delay (seconds).
# Throughput is what you actually get, and on a single TCP flow it is
# capped by  window_size / RTT  -- bandwidth never enters that formula.

LINKS = [
    # name, bandwidth in Mbps, round-trip time in milliseconds
    ("LAN, same rack",        1000,   0.5),
    ("City fibre",             100,   10.0),
    ("Transatlantic",          100,   80.0),
    ("Satellite (GEO)",         50,  600.0),
    ("Transatlantic, 10x pipe", 1000,  80.0),
]

DEFAULT_WINDOW_KB = 64   # the classic un-tuned TCP receive window


def bdp_bytes(mbps, rtt_ms):
    bits = mbps * 1_000_000 * (rtt_ms / 1000.0)
    return bits / 8.0


print("link                      Mbps    RTT      BDP   win64KB gives")
print("-------------------------------------------------------------")
for name, mbps, rtt in LINKS:
    bdp = bdp_bytes(mbps, rtt)
    # Throughput actually achievable with a fixed window:
    achievable_bps = (DEFAULT_WINDOW_KB * 1024 * 8) / (rtt / 1000.0)
    achievable_mbps = achievable_bps / 1_000_000
    capped = min(achievable_mbps, mbps)
    print(name.ljust(25) + str(mbps).rjust(5) +
          ("%.1fms" % rtt).rjust(9) +
          ("%.0fKB" % (bdp / 1024)).rjust(9) +
          ("%.1f Mbps" % capped).rjust(13))

print("")
# The window you would NEED to actually fill each pipe.
print("window required to saturate each link:")
for name, mbps, rtt in LINKS:
    need_kb = bdp_bytes(mbps, rtt) / 1024
    print("  " + name.ljust(25) + ("%.0f KB" % need_kb).rjust(9))

print("")
# The headline result: 10x the bandwidth, identical throughput.
print("'Transatlantic' and 'Transatlantic, 10x pipe' differ by 10x in")
print("bandwidth and deliver the SAME throughput with a 64KB window.")
print("Latency, not capacity, is the binding constraint there.")

# One transfer, decomposed. Propagation is distance; transmission is
# size/rate; queuing and processing are the router's fault.
SIZE_KB = 1500
prop_ms = 40.0                                   # one way, transatlantic
trans_ms = (SIZE_KB * 1024 * 8) / (100 * 1_000_000) * 1000
queue_ms = 3.0
proc_ms = 0.2
print("")
print("one 1500KB transfer over the 100Mbps/80ms link:")
print("  propagation  %.2f ms" % prop_ms)
print("  transmission %.2f ms" % trans_ms)
print("  queuing      %.2f ms" % queue_ms)
print("  processing   %.2f ms" % proc_ms)
print("  total        %.2f ms" % (prop_ms + trans_ms + queue_ms + proc_ms))

link                      Mbps    RTT      BDP   win64KB gives
-------------------------------------------------------------
LAN, same rack            1000    0.5ms     61KB  1000.0 Mbps
City fibre                 100   10.0ms    122KB    52.4 Mbps
Transatlantic              100   80.0ms    977KB     6.6 Mbps
Satellite (GEO)             50  600.0ms   3662KB     0.9 Mbps
Transatlantic, 10x pipe   1000   80.0ms   9766KB     6.6 Mbps

window required to saturate each link:
  LAN, same rack               61 KB
  City fibre                  122 KB
  Transatlantic               977 KB
  Satellite (GEO)            3662 KB
  Transatlantic, 10x pipe    9766 KB

'Transatlantic' and 'Transatlantic, 10x pipe' differ by 10x in
bandwidth and deliver the SAME throughput with a 64KB window.
Latency, not capacity, is the binding constraint there.

one 1500KB transfer over the 100Mbps/80ms link:
  propagation  40.00 ms
  transmission 122.88 ms
  queuing      3.00 ms
  processing   0.20 ms
  total        166.08 ms

RFC 7323 - TCP Extensions for High Performance — Computer Networking