Resolved incident

Region us-central1 partial degradation

Nebius · Major · Started

Duration
21h 33m
Severity
Major
Status
Postmortem

Summary

Major · Postmortem

[https://nebius.com/blog/posts/incident-post-mortem-analysis-us-central1-service-disruption-on-august-19](https://nebius.com/blog/posts/incident-post-mortem-analysis-us-central1-service-disruption-on-august-19)

View official incident ↗

Event timeline

How this incident unfolded, from the official status page.

  1. Postmortem

    Nebius

    [https://nebius.com/blog/posts/incident-post-mortem-analysis-us-central1-service-disruption-on-august-19](https://nebius.com/blog/posts/incident-post-mortem-analysis-us-central1-service-disruption-on-august-19)

  2. Resolved

    Nebius

    This incident has been resolved.

  3. Identified

    Nebius

    Most critical issues have been resolved and the region is operational, though a subset of nodes remains unavailable.

  4. Identified

    Nebius

    The Managed Service for Kubernetes is nearly fully restored, though a subset of nodes remains unavailable.

  5. Identified

    Nebius

    Most nodes are operating normally. The Managed Service for Kubernetes® is still degraded, but the situation is improving.

  6. Identified

    Nebius

    Most nodes are operating normally. The Managed Service for Kubernetes® is experiencing a partial outage, and the team is actively working to restore it.

  7. Identified

    Nebius

    Services are still gradually continuing to recover. Many of the virtual machines are already running, but not all of them.

  8. Identified

    Nebius

    Services are still gradually continuing to recover. Some virtual machines are already running, but not all of them.

  9. Identified

    Nebius

    Services are gradually continuing to recover.

  10. Identified

    Nebius

    The root cause was fixed, and the situation is stabilizing. The services are recovering.

  11. Identified

    Nebius

    We are still experiencing temperature issues in the us-central1. Team is working on it

  12. Identified

    Nebius

    The root cause has been identified as overheating hardware. GPU performance is impacted, and some nodes may be unavailable.

  13. Investigating

    Nebius

    We are currently investigating this issue.