Design a Chat Application: System Design Interview Walkthrough

Chat systems combine real-time connections, durable storage and fan-out, which is why interviewers like them. For the general approach see the system design cheat sheet.

1. Requirements

Functional: 1:1 chat, group chat, delivery and read receipts, online status, media sharing, message history. Non-functional: low latency, reliable delivery, ordering per chat, high availability, scale to millions of concurrent connections.

2. High-level architecture

Client <-> Gateway (WebSocket) <-> Chat Service <-> Message Queue
                                         |               |
                                   Session Store     Message DB
                                   (user -> server)       |
                                                    Push Service (offline)

• Gateway / connection servers hold WebSocket connections.
• A session store (Redis) maps userId -> connection server.
• Chat service validates, assigns IDs, persists and routes.
• Message queue decouples send from delivery and fan-out.
• Push service (FCM/APNs) handles offline users.

3. Message flow (1:1)

1. A sends a message over its WebSocket.
2. Server assigns a message ID and sequence, persists it, acknowledges to A ("sent").
3. Server looks up B's connection server. If online, forwards it; B acks ("delivered").
4. If offline, push notification; message waits in storage.
5. B reads; a read receipt flows back.

Use idempotent message IDs so retries don't create duplicates.

4. Ordering

Order only within a conversation. Use a server-assigned, monotonically increasing sequence per conversation (or a time-sortable ID). On reconnect the client asks for messages after its last known sequence.

5. Storage

Access pattern: append to a conversation, read recent messages. A wide-column or key-value store partitioned by conversationId and clustered by time fits well. Keep hot data fast, archive older data cheaply.

6. Presence

• Heartbeats from clients; mark offline after a timeout.
• Don't broadcast every change to every contact. Fetch on demand or subscribe only to visible contacts.

7. Group chat

• Small groups: fan out one message to each member's connection or queue.
• Large groups/channels: write once, let members pull or use hybrid fan-out.

8. Scaling issues to mention

• Millions of concurrent connections: many gateway servers, connection draining on deploy
• Server crash: clients reconnect and resync
• End-to-end encryption (keys per device) if asked
• Media: upload to object storage, send a reference in the message

Connection management at scale

A single gateway server can hold on the order of tens of thousands of WebSocket connections, depending on memory and message rate. For 10 million concurrent users you need hundreds of gateway servers behind a load balancer that supports long-lived connections.

Key details:

• Sticky routing is not needed if a session store maps each user to their current gateway.
• Heartbeats every 20-30 seconds detect dead connections and keep NATs open.
• Reconnect with backoff and jitter so a gateway crash does not trigger a thundering herd.
• Graceful deploys: drain connections slowly, asking clients to reconnect elsewhere.

Message schema

A message carries: message ID, conversation ID, sender ID, sequence number, timestamp, type (text, image, system), payload or media reference, and status. Use client-generated IDs for idempotent retries, and server-assigned sequences for ordering.

Delivery guarantees

• At-least-once delivery with client-side de-duplication by message ID
• Acknowledgements at three levels: server received, recipient device received, recipient read
• Clients keep the last received sequence per conversation and request anything newer after reconnecting

Storage design

Data	Store	Why
Messages	Wide-column / key-value partitioned by conversation	Append-heavy, range reads by time
Conversations and members	Relational or document store	Relationships, small and consistent
Presence	In-memory store with TTL	Ephemeral, high churn
Media	Object storage and CDN	Large blobs

Group chats: fan-out choices

For a group of 50, fan-out on write to each member's inbox is fine. For a channel with a million members, write the message once and let clients pull by sequence, perhaps with a push notification as a nudge. This is the same trade-off discussed in the news feed design.

End-to-end encryption (if asked)

Each device holds keys; the server stores encrypted payloads and cannot read them. Group chats need key distribution to every member's devices and re-keying when membership changes. Mention the trade-offs: server-side search and spam detection become harder.

Interview tips

Draw the send path first, then the offline path, then ordering. Many candidates forget that the server must persist before acknowledging, otherwise a crash loses acknowledged messages.

Related reading

• Design a news feed
• Design a notification system
• 12 common system design interview questions