Increased join latency in US East and US Central

Event timeline

How this incident unfolded, from the official status page.

1. Investigating

   14 days ago

   LiveKit

   We are currently investigating higher than normal join latencies affecting a small percentage of participants in our US East and US Central regions. The latency increase appears to have begun at 21:30 UTC. Affected participants may notice increased join latencies higher than 5 seconds. The SDK retries automatically, and most of these joins succeed on retry. If your application stops waiting for an agent before then, those calls may time out. We will share another update as we learn more.

2. Monitoring

   14 days ago

   LiveKit

   We identified connection drops at load balancers in our US Central and US East regions that caused a subset of connection latencies to take 5 seconds or longer. We applied a mitigation at 23:15 UTC, and we are seeing connection speeds return to normal. We are continuing to monitor before marking this resolved.

3. Resolved

   14 days ago

   LiveKit

   Between 21:30 and 23:17 UTC, roughly 10% of all WebRTC connections in our US East and US Central regions took 5 seconds or longer to establish. We applied a mitigation at 23:15 UTC, and join latencies have returned to normal by 23:17 UTC. We will follow up with more details on the root cause and preventive actions in a postmortem. We apologize for the impact and appreciate your patience.