Past incident

Authentication errors in prod-us-west-0

Grafana · Minor · Started

Duration
Unavailable
Severity
Minor
Status
No longer listed as active

Summary

Minor · No longer listed as active

The fix has been implemented, and we are monitoring the results.

View official incident ↗

Event timeline

How this incident unfolded, from the official status page.

  1. Investigating

    Grafana

    We are investigating increased authentication errors affecting Grafana Cloud in the prod-us-west-0 region. Customers may experience intermittent failures for requests that require authentication. Our engineering team has identified database performance degradation affecting the authentication service and is investigating the underlying cause. We will provide another update as more information becomes available.

  2. Identified

    Grafana

    The issue has been identified, and we have applied mitigations on our end. We are currently observing the progress.

  3. Identified

    Grafana

    We have noticed improvements across services. We will now be applying a more permanent fix. Impacted users may experience a disruption in the auth-api service of up to 15min while we apply a DB patch that will improve performance.

  4. Monitoring

    Grafana

    The fix has been implemented, and we are monitoring the results.