Elevated latency and errors in AWS us-east-1

Resolved

Minor impact

Started

Lasted 2h 17m

View official incident ↗

Updates

  1. Resolved

    This incident has been resolved. The AWS us-east-1 region is stable and all affected namespaces have been failed back.

  2. Monitoring

    Latency and error rates in AWS us-east-1 have returned to normal levels and affected namespaces have recovered. We are continuing to monitor the region. We will provide another update once failback is complete.

  3. Investigating

    Mitigation remains in progress. We are continuing to fail over affected namespaces with Multi-Region Replication enabled to their replica regions. Some customers may continue to experience elevated latency while this work is underway. We will provide another update within 30 minutes.

  4. Investigating

    We have identified the cause of elevated latency affecting a subset of namespaces in AWS us-east-1. Namespaces with Multi-Region Replication enabled are being failed over to their replica region to restore service. We will provide another update within 30 minutes.

  5. Investigating

    We are investigating elevated latency and increased deadline exceeded errors affecting some namespaces in the AWS us-east-1 region. Workflows may progress more slowly than usual. Our team is actively working to identify the cause. We will provide another update within 30 minutes.