Scaling and Load Balancing

Build the mental model

Approach	How it works
Vertical scaling	Give the existing server more CPU/memory/disk. Simple, but a hard ceiling and a single point of failure.
Horizontal scaling	Add more servers behind a load balancer. No practical ceiling, survives one instance failing — if the app can actually run as multiple copies.

Not every app can scale horizontally as-is

An app keeping session data, uploaded files, or other state in one server's memory or disk cannot be scaled horizontally without changes — a user's next request may land on an instance that never saw their data.

An app is stateless when any instance can handle any request, because all state that matters lives outside the app process — in a database, shared cache, or object storage. Stateless apps scale with almost no extra thought.

Auto-scaling watches a signal like CPU or queue depth and adjusts instance count automatically. See Kubernetes' scaling-and-hpa and AWS Fundamentals' load-balancing-and-autoscaling lessons for hands-on depth.

Horizontal Scaling: Adding more server instances and distributing traffic across them with a load balancer.

Vertical Scaling: Giving one existing server more CPU, memory, or disk instead of adding more servers.

Stateless: An app design where any instance can handle any request because important state lives outside the app process.

text

VERTICAL VS HORIZONTAL SCALING
------------------------------
VERTICAL VS HORIZONTAL SCALING
---------------------------------

VERTICAL (bigger server)         HORIZONTAL (more servers)
-------------------------        --------------------------
                                        [ LOAD BALANCER ]
    [ SMALL SERVER ]                    /      |       \
          |                            v       v        v
     resize up                    [ app-1 ] [ app-2 ] [ app-3 ]
          v
    [ BIG SERVER ]                 traffic spread across
    single point of failure        instances; one instance
    hard ceiling                   can fail without an outage

Connect it to a real scenario

Round robin cycles through instances in order, giving each an equal share of requests.

Running five requests through three mock instances shows the wraparound clearly — the fourth request lands back on the first instance.

This simulation assumes statelessness

If one instance were the only one holding a user's session in memory, round robin would silently break that user's experience the moment their next request landed elsewhere.

Vertical scaling resizes one server; horizontal scaling adds more servers behind a load balancer — each has real tradeoffs.
Stateless apps (no reliance on local in-memory state) scale horizontally far more easily than stateful ones.
Auto-scaling automates horizontal scaling by reacting to a live signal like CPU or queue depth instead of a human guess.

Try the working example

javascript

function createRoundRobinBalancer(instances) {
  let next = 0;
  return function pickInstance() {
    const instance = instances[next];
    next = (next + 1) % instances.length;
    return instance;
  };
}

const instances = ["app-1", "app-2", "app-3"];
const pick = createRoundRobinBalancer(instances);

const requests = ["req-A", "req-B", "req-C", "req-D", "req-E"];
for (const req of requests) {
  console.log(`${req} -> ${pick()}`);
}

You should see

req-A -> app-1
req-B -> app-2
req-C -> app-3
req-D -> app-1
req-E -> app-2

5-minute try-it

Modify createRoundRobinBalancer to accept a weights array (e.g. app-1 should get twice the traffic of app-2 and app-3). Run the same five requests and check the distribution matches the weights.

One important caution

Adding more instances behind a load balancer while the app still keeps session state in local memory, causing random login/cart-loss bugs.

Treating vertical scaling as a permanent fix instead of a stopgap — it always hits a ceiling, usually at the worst possible time.

Which app scales horizontally more easily?

App A stores each user's session data in server memory. App B stores session data in a shared database/cache all instances can read. Which one scales horizontally more easily?

A. App A, because memory is faster than a databaseB. App B, because any instance can serve any user's session — the instances are interchangeableC. They scale equally wellD. Neither can scale horizontally

MDN — Load balancer — Cloud & Deployment