Elevated webhook delivery latency

Event timeline

How this incident unfolded, from the official status page.

1. Investigating

   29 days ago

   Mux

   We are currently investigating elevated webhook delivery times. Beginning at 16:40 UTC, some webhooks are being delivered with a delay of approximately 5 minutes. Video processing, playback, and API requests are not affected, and no webhook events are being dropped — delayed events will still be delivered. We will provide an update as soon as we have more information.

2. Identified

   29 days ago

   Mux

   We have identified the cause of the elevated webhook delivery times. An issue in one of our internal database clusters introduced a delay in the pipeline that queues webhook events, resulting in delivery delays beginning at 16:40 UTC. Video processing, playback, and API requests are unaffected, and no events have been lost — delayed webhooks are still being delivered. Delivery times are beginning to recover.

3. Monitoring

   29 days ago

   Mux

   Webhook delivery times have returned to normal. Between 16:40 UTC and 17:36 UTC, webhook events were delivered with delays averaging around 5 minutes, with some events delayed up to 15 minutes. Video processing, playback, and API requests were unaffected, and no events were lost — all delayed webhooks have been delivered.We have addressed the underlying issue in one of our internal database clusters and are monitoring to confirm.

4. Resolved

   29 days ago

   Mux

   This incident has been resolved. Between 16:40 UTC and 17:36 UTC, webhook events were delivered with delays averaging around 5 minutes, with some events delayed up to 15 minutes. Video processing, playback, and API requests were unaffected, and no events were lost — all delayed webhooks have been delivered.The underlying issue in one of our internal database clusters has been addressed, and delivery times have been stable since 17:36 UTC. We apologize for the disruption.