Real-time Updates Experiencing Delays

Resolved

Started

Lasted 11h 19m

View official incident ↗

Updates

  1. Resolved

    After performing the rollbacks and monitoring traffic and system health for a few hours, we have not witnessed any more issues with real-time updates, and services are restored.

  2. Monitoring

    The team has performed a rollback of the notifier service on our main cluster and are monitoring to confirm restored performance and functionality.

  3. Identified

    Our previous fix helped but wasn’t reliably working so we are working on a rollback of our notifier service on dedicated environments

  4. Monitoring

    Rolled out a fix and are monitoring to confirm.

  5. Investigating

    We've received a few more reports of delayed real-time updates, and the response team is continuing to investigate.

  6. Resolved

    We are no longer observing delays with real-time updates, and our systems have stabilized.

  7. Monitoring

    The response team has pushed out a fix, and we will continue to monitor the situation as our systems stabilize.

  8. Identified

    We've identified the issue with real-time updates and the response team is pushing out a fix.

  9. Investigating

    We're currently investigating reports of delays regarding real-time updates.