We are experiencing a major outage in our US2 cluster

Resolved

Critical impact

Started

Lasted 5h 6m

View official incident ↗

Updates

  1. Resolved

    This incident has been resolved.

  2. Monitoring

    We are continuing to monitor for any further issues.

  3. Monitoring

    A fix has been implemented, and we are monitoring the results. WebSocket connections should now be functional.

  4. Monitoring

    The team has been able to identify the issue and push a fix. We are seeing a full recovery of the services. We will continue to monitor.

  5. Investigating

    Our team is still investigating the issue within our infrastructure that is preventing us from adding WebSocket capacity to the US2 cluster. This issue is specifically impacting our WebSocket deployments. The majority of clients are currently unable to connect using WebSocket. However fallback mechanisms (SockJS, HTTP streaming, and HTTP polling) are available for connection if they are not disabled in your configuration.

  6. Investigating

    We are escalating the current issue from a partial outage to a major outage. The majority of traffic in the US2 cluster is currently affected. Our team is fully engaged and working to resolve the issue.

  7. Investigating

    We are investigating what is causing the issue to our US cluster.