Scaling and Pub/Sub
A single Node process can hold a surprising number of open WebSockets — tens of thousands on a healthy machine. So for a while, scaling feels like a problem you can ignore. Then traffic grows, you add a second server behind a load balancer, and something strange happens: a message sent by a user on server A never reaches a user on server B. The chat looks broken even though every line of your code is correct.
This module is about that gap and how to close it.
A connection is a thing that lives in memory
Section titled “A connection is a thing that lives in memory”The detail that makes scaling hard is easy to miss. When a client connects, the server keeps an in-memory object for that socket — the file descriptor, any buffered frames, and usually some application state you attached, like the user’s id or which rooms they joined.
That object lives in one process, on one machine. It is not in a database. It is not shared. The other server has never heard of it. If that process restarts, every connection it held is gone and every client has to reconnect.
So “the list of who is connected” is really “the list of who is connected to this specific instance”. And that single fact is what breaks broadcasting.
Broadcasting only reaches this instance
Section titled “Broadcasting only reaches this instance”Imagine the simplest real-time feature: someone posts a chat message and you want everyone to see it. The natural code is “loop over my connected clients and send to each”.
The problem is the word my. Each instance can only loop over the sockets it personally holds. When clients are spread across two instances by a load balancer, a broadcast on one instance reaches half the room and silently skips the other half.
flowchart TB
lb["Load balancer"]
subgraph A["Instance A (in-memory sockets)"]
a1["client 1"]
a2["client 2"]
end
subgraph B["Instance B (in-memory sockets)"]
b1["client 3"]
b2["client 4"]
end
lb --> a1
lb --> a2
lb --> b1
lb --> b2
a1 -. "posts a message" .-> A
A == "broadcast reaches A only" ==> a2
A -. "client 3 and 4 never hear it" .-x B Client 1 posts a message. Instance A faithfully delivers it to client 2, who is also on A. Clients 3 and 4 are on instance B, which has no idea the message ever existed. From their seats, the room went quiet. Nothing crashed — the message simply had no path between instances.
What this module covers
Section titled “What this module covers”Solving this is mostly an architecture exercise, so the lessons lean on diagrams and small TypeScript sketches rather than a single runnable server. The path:
- Why scaling is hard — connection state in memory, the per-connection cost, balancing long-lived connections, and the cross-instance broadcast problem stated precisely.
- A pub/sub backplane — put a shared bus (Redis pub/sub, NATS, Kafka) between instances so every published message reaches every instance, which then fans out to its own local clients. With an in-page demo of two instances sharing one bus.
- Presence across instances — tracking who is online when “online” is scattered across machines: a shared store, heartbeats, expiry, and the reconnect stampede.
- Sticky vs stateless — route a client back to its instance, or let any instance serve it from shared state, and how to configure the load balancer either way.