Resolved incident

Standard and Basic cluster creation on AWS delayed

Cockroach Labs · Critical · Started

Duration
6h 16m
Severity
Critical
Status
Resolved

Summary

Critical · Resolved

All cluster operations are working normally.

View official incident ↗

Event timeline

How this incident unfolded, from the official status page.

  1. Investigating

    Cockroach Labs

    We are investigating errors that are causing delays in creation of Basic and Standard clusters on AWS. Basic and Standard cluster creation on GCP is operating normally at this time.

  2. Investigating

    Cockroach Labs

    We are investigating errors with Basic and Standard clusters on AWS. Cluster creation, certain cluster edit operations (such as adding a region), and certain CockroachDB Cloud Console functionality (such as listing databases or users) are also impacted. Basic and Standard clusters on GCP are operating normally at this time.

  3. Identified

    Cockroach Labs

    We have identified the cause of cluster creation failures impacting Basic and Standard AWS clusters. As a precautionary measure, while we apply a fix, we have disabled all cluster creation in Cockroach Cloud. We will restore cluster creation when it is safe to do so. There is no SQL availability impact to any existing clusters. We will update here by 3:00 PM US/Eastern with more information.

  4. Identified

    Cockroach Labs

    We are continuing to work on a fix for this issue. Next update will be by 4:00 PM US/Eastern time.

  5. Identified

    Cockroach Labs

    We are continuing to work on a fix for this issue. Next update will be by 4:30 PM US/Eastern time.

  6. Monitoring

    Cockroach Labs

    We have resolved the underlying issue that was preventing cluster creation and other cluster operations. We have re-enabled cluster creation on all clouds. We will monitor the situation to ensure that everything continues to operate normally.

  7. Resolved

    Cockroach Labs

    All cluster operations are working normally.