Elevated latency and errors in AWS us-east-1

Event timeline

How this incident unfolded, from the official status page.

1. Investigating

   14 days ago

   Temporal

   We are investigating elevated latency and increased deadline exceeded errors affecting some namespaces in the AWS us-east-1 region. Workflows may progress more slowly than usual. Our team is actively working to identify the cause. We will provide another update within 30 minutes.

2. Investigating

   14 days ago

   Temporal

   We have identified the cause of elevated latency affecting a subset of namespaces in AWS us-east-1. Namespaces with Multi-Region Replication enabled are being failed over to their replica region to restore service. We will provide another update within 30 minutes.

3. Investigating

   14 days ago

   Temporal

   Mitigation remains in progress. We are continuing to fail over affected namespaces with Multi-Region Replication enabled to their replica regions. Some customers may continue to experience elevated latency while this work is underway. We will provide another update within 30 minutes.

4. Monitoring

   14 days ago

   Temporal

   Latency and error rates in AWS us-east-1 have returned to normal levels and affected namespaces have recovered. We are continuing to monitor the region. We will provide another update once failback is complete.

5. Resolved

   14 days ago

   Temporal

   This incident has been resolved. The AWS us-east-1 region is stable and all affected namespaces have been failed back.