Cut cross-region read latency from ~280 ms to under 10 ms with geo-sharded MongoDB
Built with Node.js · Express · Socket.IO · MongoDB · Mongoose
Overview
I designed and built the backend for a real-time AI chat product: LLM responses streamed to clients in chunks over WebSockets, with every message and stream chunk persisted for history. The system runs geo-distributed across five regions on four continents (Frankfurt, Montreal, Las Vegas, São Paulo, and Sydney), so a user anywhere in the world talks to infrastructure close to them.
It is built on Node.js, Express, and Socket.IO, backed by MongoDB through Mongoose; generation is delegated to a separate LLM service, and a MongoDB-backed QA cache answers repeated questions without calling the LLM at all.
The problem
A single central database meant transcontinental users paid for every read with a cross-ocean round-trip: hundreds of milliseconds on a product that is supposed to feel instant. I needed each user's chat history to live physically near them, to survive a full-region outage, and to stay inside its region for data residency, all without collapsing into one fragile central cluster.
What I built
Each region runs its own three-node MongoDB replica set (fifteen nodes in total), with write concern w=2, a 5-second write timeout, and readPreference=nearest so reads are served from the closest node. The hot collections, chat_history and chat_stream, are geo-sharded on a compound shard key of { region, regionKey }, which pins each user's data to the shard in their nearest datacenter. History and stream reads are region-scoped, so each instance only touches its own shard, and generated chunks are relayed to the client over Socket.IO.
The hard part was migration. Users occasionally get routed to a different region than the one their history lives in, after a failover, travel, or CDN routing, so their documents have to move by changing the region field. But region is part of the shard key, and a multi-document update (updateMany) cannot change a shard-key field on a sharded collection, so the migration failed. The fix was to migrate document by document with per-document save(): a single-document retryable write is the only way to move a document to another shard, guarded to skip anyone already in their home region.
const chatHistorySchema = new Schema( { user_id: { type: String, required: true }, region: { type: String, default: process.env.REGION }, regionKey: { type: Number, default: 1 }, thread_id: { type: String, required: true }, message: { type: String, default: null }, // ... }, { collection: 'chat_history', shardKey: { region: 1, regionKey: 1 }, },); // region is in the shard key, so a bulk updateMany()// can't move a user's data. Migrate per-document;// each save() is a retryable write.chatHistory.map(async chat => { chat.region = process.env.REGION; chat.regionKey = 1; await chat.save();});The results
Reads for transcontinental users dropped from roughly 250 to 300 ms down to under 10 ms, about a 28× improvement, by serving from the local shard instead of a cross-ocean round-trip. Normal requests make zero cross-region round-trips, 100% of a user's chat data stays resident in their region, and the system scales horizontally across five independent clusters rather than one central database. If an entire region goes down, the others keep serving with no data loss.