Resolved incident
Elevated Latency and Errors for Grafana Cloud Logs in AWS US East (VA)
Grafana · Minor · Started
- Duration
- 2h 59m
- Severity
- Minor
- Status
- Resolved
Summary
Minor · Resolved
Between 12:09 and 15:39 UTC on September 30, Grafana Cloud Logs in AWS US East (VA) experienced intermittent elevated latency and a small number of failed requests. Impact has returned to normal levels and all services are operating as expected. Thank you for your patience.
Event timeline
How this incident unfolded, from the official status page.
Investigating
Grafana
We are investigating elevated latency and a small number of failed requests for Grafana Cloud Logs in AWS US East (VA). Some customers writing logs to this region may see delayed ingestion or occasional write errors. We are actively investigating and will share more as we learn it. Next update by 13:38 UTC.
Investigating
Grafana
We are continuing to investigate this issue.
Identified
Grafana
We have identified a contributing factor to the elevated latency and are rolling out a mitigation to increase write capacity in AWS US East (VA). We continue to investigate a related, longer-running pattern. Next update by 14:40 UTC.
Identified
Grafana
An additional brief recurrence affected a small number of requests after our mitigation was deployed, though impact was smaller than earlier today. We continue to monitor and investigate a related, longer-running pattern. Next update by 15:40 UTC.
Resolved
Grafana
Between 12:09 and 15:39 UTC on September 30, Grafana Cloud Logs in AWS US East (VA) experienced intermittent elevated latency and a small number of failed requests. Impact has returned to normal levels and all services are operating as expected. Thank you for your patience.