Disruption affecting the provisioning of new instances in Singapore and partially in Oregon

Resolved

Major impact

Started

Lasted 12h 14m

View official incident ↗

Updates

  1. Resolved

    We've confirmed that our fix has addressed the issue and workloads are now scheduling normally in all regions.

  2. Monitoring

    Provisioning is recovering for free and paid services across all regions. Free services have resumed serving traffic. We are continuing to monitor for further issues.

  3. Monitoring

    Paid services are continuing to see recovery across builds and deploys. Free services remain disabled and will return errors if loaded in a browser. Singapore and part of Oregon remain impacted.

  4. Monitoring

    We are continuing to monitor for any further issues.

  5. Monitoring

    We are disabling new free services in Singapore and partially in Oregon.

  6. Monitoring

    We're seeing recovery across all regions.

  7. Identified

    We are continuing to work on a fix for this issue.

  8. Identified

    We are now seeing the same issue affecting services in Oregon partially. We are investigating to remediate this as quickly as possible.

  9. Identified

    Services with paid compute should now be operating normally. Please note that free services in this region are still intermittently disabled.

  10. Investigating

    We are continuing to investigate this issue.

  11. Investigating

    We are seeing some recovery following our mitigation efforts, but we are still working to identify the root cause for a long-term resolution.

  12. Investigating

    We have temporarily disabled free services in Singapore. We are continuing to investigate and work to mitigate the issue.

  13. Investigating

    This issue may affect new instances during builds, scaling, and similar operations. You may experience build/deploy failures. We are still investigating.