Degraded performance for Grafana Cloud Metrics in prod-us-central-7

Resolved

Major impact

Started

Lasted 1h 36m

View official incident ↗

Updates

  1. Resolved

    Increased latency on a cloud provider database resulted in a 50m write and rule evaluation outage, starting about 2020 UTC. Metrics rule evaluations were last to recover, around 2110 UTC.

  2. Monitoring

    Traffic has recovered and we are monitoring the results.

  3. Investigating

    We are continuing to investigate this issue.

  4. Investigating

    We are continuing to investigate this issue.

  5. Investigating

    We are investigating an issue affecting Grafana Cloud Metrics, Logs, and Traces in the prod-us-central-7 region. Some customers may see failed or delayed writes, query errors, and failed alerting and recording rule evaluations.