Building Production-Ready Voice Rooms with LiveKit: The Bugs Beyond "Hello World"

Getting two people talking in a voice room is the easy 10%. I proved that to myself in an afternoon with LiveKit's quickstart. The other 90% — the part where the room survives flaky phone networks, background app kills, duplicate webhooks, and a host who taps "leave" three times because the UI didn't update fast enough — took a lot longer, and most of what I learned isn't in any tutorial.

I recently built live voice channels for a community platform on this stack:

• LiveKit for WebRTC media and room infrastructure
• Go for the backend API
• PostgreSQL for durable session history and usage accounting
• Redis for short-lived coordination
• React Native / Expo for the mobile client
• TypeScript for the client SDK and shared app layer

The brief sounded simple. Hosts start a call and land inside it immediately. Members join, listen, react, raise a hand. Moderators invite speakers, mute people, remove people. Ending a call kicks everyone out. Old calls stick around in the channel history.

None of that is hard to describe. What's hard is keeping the application's idea of "a call" in sync with LiveKit's idea of "a room" — especially once packets, webhooks, and requests start arriving late, twice, or not at all. This post is a walk through the bugs that taught me that lesson, roughly in the order I hit them.

Three sources of truth, on purpose

Before any of the specific bugs make sense, it helps to know how I split the state. I didn't want one system trying to be everything, so I gave each layer a narrow job.

PostgreSQL holds the facts that need to survive a restart — the voice session's identity, which channel and community it belongs to, who's hosting, when it started and ended, the speaking policy, peak attendance, participation intervals, why it ended, and a state version number. This is also where billing and history come from.

Redis holds state that matters right now but doesn't need to outlive the call — a user's active room lease, a connection generation counter, the raised-hand queue, temporary speaking grants, reaction dedup keys, rate limits, start cooldowns, budget-alert suppression.

LiveKit owns the media plane — audio tracks, room membership, participant permissions, reconnection, removal, and server-to-client data packets.

The backend stays the authority on who can start, join, speak, moderate, or end a call. LiveKit tokens are just the receipt for a decision the application already made — they never make the decision themselves. Put simply:

PostgreSQL = durable truth
Redis      = coordination and leases
LiveKit    = live media and presence
Client     = a projection of all three, kept in sync

That separation sounds obvious in hindsight. It wasn't obvious on day one, and every time I was tempted to skip it and let LiveKit's room state stand in for application state, I paid for it later.

Bug #1: "Start call" doesn't mean "join call"

The first thing users noticed was dumb but telling: tapping "Start call" created the room just fine, and then dropped the host on a screen where they still had to tap "Join call." Nobody thinks of those as two steps. To a user, "start" means "start it and put me inside," full stop — the difference between an API resource and a WebRTC connection is my problem, not theirs.

So the client flow became: check whether this device is already in another call, create the server-side session, request a LiveKit token, connect, then render the connected screen.

The annoying part is that step two and step three can't be one atomic transaction. The session gets created in the database, and then the join to LiveKit can fail for a dozen reasons that have nothing to do with the database — the OS just denied microphone permission, the network dropped, LiveKit itself hiccuped, the user is already in a different call, the browser choked on part of the media setup.

Do nothing about that, and you get a live room with nobody in it, quietly ticking away until its idle timeout finally kills it. The fix was two rules: refuse to start a second room if the client already has an active call, and if the room gets created but the automatic join fails, immediately tear the new session back down. It's a small thing, but it's the first time I really internalized that distributed workflows are rarely atomic, and when they can't be, you owe them explicit cleanup.

Bug #2: the zombie call

This one was the most confusing to debug because everything about it looked like it should already work.

When a call ended, the server deleted the LiveKit room, which disconnected the client's WebRTC connection. LiveKit's runtime state flipped from connected to idle — correctly. But the application's call provider, sitting one layer up, still had a reference to the old call object and had no reason to let it go.

So the screen kept showing a live call. Tapping "Join" didn't fix it either, because the join function's first move was to check "is this user already associated with this channel's call?" — and since the provider still thought yes, it bailed out early and did nothing.

Server:   session ended
LiveKit:  room deleted
Runtime:  disconnected
Provider: still holding the call
UI:       still showing "live"
Tap Join: early-returns, does nothing

The actual bug was that I'd conflated two different questions: "is this call associated with my channel" and "is my transport actually connected." Those aren't the same thing. A call can be associated with a channel while the connection underneath it is joining, connected, reconnecting, idle, or failed — and "connected, as far as the UI is concerned" should only be true for the first three of those.

Once I split the two, the fix followed naturally: whenever LiveKit lands in a terminal disconnected state, the client does one authoritative read of the session. A 404 means the call really is over, so local state gets cleared. A live session still on the server means it's a real rejoin, not a stale UI — and a rejoin always requests a fresh token instead of hitting that early-return guard. One conceptual distinction, several bugs gone.

Bug #3: leaving on purpose looks identical to a network drop

Related problem: when someone deliberately hit "leave call," the LiveKit disconnect event fired — and it's the exact same event that fires when the network just falls over. My recovery logic, which existed to catch real drops, reacted to the intentional leave too and fired off an unnecessary getActive request while the leave was still in flight.

It wasn't actively harmful, since other guards kept the call from resurrecting itself. But "unnecessary request racing with an in-flight state change" is exactly the shape of bug that turns harmless today and flaky next Tuesday. I added an explicit flag around intentional leaves:

leaving = true
disconnect from LiveKit
notify the backend
clear local call state
leaving = false

The recovery effect now ignores disconnect transitions while leaving is true. The lesson generalizes past voice calls: "disconnected" is an event, not a business fact. The system needs to know why, or it'll guess wrong under load.

Bug #4: the mini-player kept opening the same room twice

The global mini-call bar lets people browse the rest of the app while staying in a voice room — tap it, and you jump back to the call. Except the first version always pushed a fresh voice screen onto the navigation stack, even when the user was already looking at that exact channel's voice screen. Two stacked copies of the same screen, weird back-button behavior, the works.

The fix was mostly about navigation hygiene: hide the mini-player entirely when its screen is already the active one, use navigation that replaces/reuses the destination instead of blindly pushing a new one, and label the button "Return" when connected versus "Rejoin" when disconnected. Small stuff, but it's the kind of small stuff that becomes very visible once an audio connection can outlive the screen that started it.

Bug #5: reactions that only the sender could see

This was the funniest bug to reproduce and the most obvious once I saw it. Tap the 👏 reaction, the emoji animates on your own screen, everything looks perfect — and nobody else in the room sees a thing.

Turns out the reaction was 100% local UI. The client played its animation, sent the request to the server, the server accepted and logged it… and then never told anyone else. The fix was to actually wire the broadcast:

client sends reaction
    → backend authorizes + rate-limits it
    → backend broadcasts a LiveKit data packet
    → every connected client receives it
    → every client plays the animation

This is also where I had to think about delivery guarantees. Reactions are throwaway by nature — missing an occasional 👏 is fine, and replaying an old one after the fact can feel worse than just dropping it. So reactions go out as lossy packets. Room state changes are the opposite: a raised hand, a speaker promotion, a removal that never reaches someone's client can leave their UI wrong indefinitely, so those go out as reliable packets. LiveKit's own docs are blunt about the distinction — reliable packets are ordered with limited retransmission, lossy ones are fire-and-forget with no ordering guarantee at all, and neither is durably stored for a client that's momentarily offline. (LiveKit data packet docs)

Which is really just:

• Reactions and visual flourishes → lossy
• "Something changed, go check" hints → reliable
• The actual facts → HTTP API + database

Reliable isn't durable, and that bit me too

Switching state-change packets to reliable delivery fixed the packet-loss problem I was actively fighting, but it didn't fix the underlying issue: a client that's disconnected for even a second when a packet fires will still miss it, reliable or not. LiveKit says as much — reliable delivery is best-effort, not stored for offline receivers. (source)

So every voice session carries a monotonically increasing state_version, and the packets don't even try to carry the full room state — they're just a nudge:

{
  "type": "voice.state.changed",
  "event": "hand_raised",
  "session_id": "...",
  "state_version": 12
}

The client compares the version it just received to the one it has, and only re-fetches when the number actually moved. That one change also killed a much dumber problem I'd built earlier: both the voice screen and the global provider were separately polling the session every ten seconds, and each poll asked LiveKit for the full participant list — so load scaled with connected clients, for no good reason. Now: disconnected viewers poll (rarely), connected clients react to packets, a terminal disconnect triggers exactly one authoritative read, and manual refresh is always there as an escape hatch. The API stays the one source of truth; the packets are just a "hey, look again" bell.

Raised hands don't care about the speaking policy

Small but easy to get wrong: a "raise hand" button shouldn't vanish just because the room's current policy already lets everyone speak. The gesture still means something — someone doesn't want to interrupt, wants to be explicitly invited in, or the policy might flip mid-call, or they simply can't publish audio yet for some unrelated reason. So the raised-hand queue lives independently of the speaking-policy setting, stored in Redis as an ordered set (repeated raises are idempotent, and moderators see a stable order, not a shuffled one).

When a moderator approves someone, the server does five things in order: grant temporary speaking permission, pop them off the raised-hand queue, push the LiveKit permission update, bump the state version, broadcast the reliable packet. The client never decides who's allowed to talk — it only ever renders whatever capability the server handed it.

The reconnect bug that quietly inflated our bill

This is the one I'm least proud of, and the most instructive.

Every connection gets a "generation" — a counter that increments each time a user (re)joins. The reason it exists: LiveKit's participant_left webhook can arrive after that same user has already reconnected, and without a generation check, that stale webhook will happily close the wrong participation record — the new one, not the old one.

My first version tried to be clever and preserve the original joined_at timestamp across a reconnect of the same session. That interacted badly with an older uniqueness constraint in the database, and the failure mode was subtle: the new generation's insert got silently ignored (it collided with the constraint), the old generation's stale leave event got correctly rejected as stale — and the old participation row just... stayed open. Which meant it kept accumulating duration straight through the network gap where the user wasn't even connected. Multiply that across a month of real usage and the RTC-minutes bill was quietly, consistently wrong.

Generation A: joined ───── left
                        (network gap)
Generation B:                  joined ───── left

The fix was to stop trying to be clever and just treat every reconnect as its own interval. On reconnect, the server now replaces the Redis lease with a new generation, hands back the old lease so it can be closed, closes the old durable participation interval, issues a token stamped with the new generation, and opens a fresh participation row when that new join is actually observed. The database's uniqueness constraint moved from being timestamp-scoped to generation-scoped. It's a good reminder that idempotency and stale-event protection are two separate concerns that have to be designed together — either one alone can look completely correct in isolation while their interaction quietly corrupts your numbers.

Webhooks are evidence, not a guarantee

Joins and leaves arrive as LiveKit webhooks, and I initially treated that stream the way I'd treat any well-behaved event feed — reliably ordered, eventually delivered. LiveKit's own docs say otherwise: delivery is retried, but never guaranteed. (source)

So every signed webhook first lands in an inbox table — event ID, type, payload hash, raw payload, attempt count, processed-at, last error — and processing is idempotent against that ID, so duplicates can't double-create participation rows. But an inbox only helps for events that show up at all. A participant_joined webhook that never arrives at all still quietly under-counts peak attendance and never opens a participation interval in the first place.

The fix was a reconciliation job that compares three views of the world: who LiveKit says is actually in the room, who Redis says holds a valid connection lease, and who Postgres says has an open participation row. Anyone present in LiveKit with a valid lease gets their participation row idempotently ensured — a no-op if the webhook already did its job, a repair (plus a peak-count correction) if it didn't. Anyone whose lease outlives the join grace period with no matching presence gets their interval closed. Webhooks give you low latency; reconciliation gives you eventual correctness. In practice you need both, and neither one covers for the other.

Ending a call is a workflow, not an API call

I originally thought DeleteRoom was the finish line. It's actually step five of nine: authorize the actor, atomically flip the session from live to ending, close open participation intervals, clear ephemeral grants and raised hands, delete the LiveKit room, mark the durable session ended, record why it ended, keep history and peak attendance intact, and — this is the part that matters — be able to recover if the process dies in the middle of all that.

That intermediate ending state earns its keep the day the process crashes between the database transition and the LiveKit deletion call. A recovery job can find sessions stuck in ending and finish the job. Without that state sitting there as breadcrumb, a retry has no way to tell whether the original attempt never started, half-finished, fully finished, or finished remotely but failed to record it locally. Any operation that spans more than one system deserves an explicit "in progress" state — it's cheap insurance against exactly this class of crash.

Audio isn't one codebase's problem — it's two

The web client and the native client share the same voice provider at the top level, but the audio lifecycle underneath diverges completely once you get into platform specifics.

On web: remote tracks need to be attached to actual <audio> elements, browser autoplay policy can silently block playback (hence an explicit "Enable audio" button for when it does), and microphone access is gated by both permission prompts and secure-context requirements.

On React Native: the native LiveKit audio session has to start and stop in lockstep with the call itself, Android needs a foreground service for any sustained mic use, and iOS needs real background-audio configuration to keep voice alive when the app isn't foregrounded. One decision I made deliberately: I did not declare iOS's voip background mode, because that entitlement implies a real PushKit/CallKit implementation behind it, and declaring it without one is a good way to draw unwanted App Review attention. Plain background audio mode covers the actual requirement without that risk.

The practical takeaway: a browser is a fine place to validate signalling and basic audio attach/playback, but it will never tell you whether an Android foreground service survives being backgrounded, or whether an iPhone resumes cleanly after a phone call interrupts your app's audio session. That testing has to happen on real devices, full stop.

Don't make members hit an admin endpoint just to ask "can I do this?"

Smaller bug, same category of lesson: the client was calling an administrative permissions endpoint to decide whether the current user could start a call. Ordinary members correctly got 403 Forbidden back — but the UI never needed the whole permission-management console, just a yes/no answer about the person actually looking at the screen.

The fix was a member-safe endpoint that just returns booleans:

{
  "startVoice": false,
  "manageVoice": false,
  "endVoice": false
}

The general rule I took away from this: don't send a client through an admin resource to answer a narrow self-service question. If "can I do X" is a common question, it deserves its own narrow "my effective permissions" shape.

Error codes are part of the contract, and they drift

At one point the client was checking for VOICE_ALREADY_LIVE. The API had actually shipped VOICE_SESSION_ALREADY_LIVE. The call still failed correctly — but the specific, helpful recovery message the UI was supposed to show never fired, because the string comparison silently missed.

That's not a transport bug, it's contract drift, and regenerating the TypeScript client from the OpenAPI spec surfaced more of it than I expected — the voice SDK had its own hand-rolled session types and was building request paths by hand instead of going through the generated client. Moving it onto the generated client fixed the class of bug entirely: paths are now compiler-checked, request bodies come straight from the spec, parameter names are generated instead of typed by hand, response shapes derive from generated schemas, and required fields get validated right at the SDK boundary.

One edge case the generated client got wrong on its own: the reaction endpoint returns a plain 202 Accepted with a text body, and the client assumed JSON until I explicitly told that one request to parse as text. Generated clients kill a lot of drift, but they still need real tests against how the server actually responds, not just what the spec says it should return.

Rate limiting and idempotency have to be one atomic step

Reactions are protected by both a token-bucket rate limit and an event-ID dedup key for retries. My first pass ordered these as: store the event ID, check the rate limit, reject if the bucket's empty. Which meant a request that got rejected for being rate-limited still consumed its own dedup key — so a legitimate retry with the same event ID got silently swallowed as "already seen," instead of being fairly re-evaluated.

The fix was to fold both checks into a single Redis script: bail out immediately if this event ID already succeeded before, refill and check the token bucket, and only if there's a token available, consume it and store the dedup key together. If the attempt fails for any reason, it must not leave behind any trace that looks like a success. That rule isn't specific to reactions at all — it's the same rule that matters for payments, webhook processing, file uploads, or any other command a client might legitimately retry.

Capacity limits aren't an afterthought, they're the budget

Voice calls cost real money per minute, so I built the guardrails in up front rather than bolting them on later: a global cap on concurrent sessions, a per-room participant cap, per-user start-attempt limits, per-community start cooldowns, timeouts for empty rooms and for rooms whose host never shows, monthly RTC-minute accounting, and budget thresholds that eventually cut off new starts entirely.

Starting a room grabs a global transactional advisory lock before checking capacity, which deliberately serializes every room creation around that one limit. That's a bottleneck waiting to happen at very high room counts — but at the scale this actually runs at, it buys a simple, obviously-correct invariant (live rooms + ending rooms <= configured capacity) for free. I'd rather ship the correct-but-simple version and replace the lock with something more distributed once real numbers say I need to, than guess at a fancier design today.

What actually caught bugs in testing

The happy-path join test was the least useful test I wrote. The ones that actually caught real bugs simulated the failure modes above directly: two communities racing for the last room slot, two hosts racing to start a room in the same channel, duplicate webhook delivery, a failed webhook retried later, a stale leave event from an old connection generation, duplicate joins, reconnects that should produce two separate participation intervals (not one polluted one), live usage math for intervals that are still open, historical ordering and duration, reaction rate-limit and retry behavior, reliable-packet parsing, malformed or unknown packet types, the plain-text 202 response, full generated-contract type checking, and migration cleanliness.

The layering that actually worked: unit tests, then database integration tests, then SDK transport tests, then multi-participant browser tests, and only real iOS and Android devices at the top. A browser will happily prove signalling and remote-audio attachment work. It has no opinion on whether an Android foreground service survives being backgrounded, or whether an iPhone's audio session recovers gracefully after a real phone call interrupts it.

What I'd carry into the next one

A few things stuck with me past this specific project.

The media server is not your database. LiveKit knows who's connected right now. It has no opinion on billing, audit history, or who was authorized to do what — that's still entirely on you.

"Connected" isn't a boolean. Joining, connected, reconnecting, intentionally leaving, unexpectedly dropped, and ended are six different states with six different correct responses. Collapsing them is where the zombie-call bug came from.

Reconnects need generations. User identity by itself isn't enough once events from two different connection attempts can be in flight at the same time.

Webhooks need a reconciliation job standing behind them. Retries help. Retries are not a guarantee. Compare what the provider says to what you actually recorded, on a schedule.

Pick packet reliability by what the packet means, not by habit. A missed 👏 is nothing. A missed moderation action is a bug someone will report.

Realtime packets should say "go look," not "here's the truth." The moment a packet tries to be the state instead of pointing at it, you've built a second, worse database.

Distributed workflows need compensation, not hope. If step one succeeds and step two fails, something has to clean up step one — nobody else is going to.

Generated contracts don't replace tests against the real server. Types check what the spec promised. Only a real request against a real response catches the server that promised JSON and shipped plain text.

None of this shows up in a WebRTC "hello world," because a hello world doesn't run for a month, doesn't get network-interrupted mid-call, and doesn't have a webhook arrive twice. A production voice room is a small distributed system wearing a microphone icon — a database, a coordination store, an RTC provider, an authorization layer, a generated client contract, a couple of very different native audio runtimes, and a pile of participants on networks you don't control, all trying to agree on one shared fact: who's actually in the room right now. The happy path was never the hard part. The hard part is the Tuesday afternoon where an old webhook shows up after a reconnect, a reliable packet misses a client that blinked offline for half a second, a room gets created but the host's join fails, and the UI is still cheerfully showing a call that ended thirty seconds ago.