Vitess database availability issues in GCP northamerica-northeast1

Resolved

Started

Lasted 43m

View official incident ↗

Updates

  1. Resolved

    Problem has been resolved. We saw three nodes fail across three availability zones simultaneously. Impacting databases hosted on those nodes. All databases have been moved to healthy nodes and we're investigating how we can further reduce impact from similar events in the future.

  2. Monitoring

    Impacted databases are recovering, we're continuing to monitor.

  3. Identified

    We're working on a fix for the impacted databases.

  4. Investigating

    We are currently investigating impact.