Resolved incident
Error rates increasing
BuildKite · Minor · Started
- Duration
- 20m
- Severity
- Minor
- Status
- Resolved
Summary
Minor · Resolved
Additional capacity was added to our redis caches. This triggered a failover between UTC 15:10 - 15:14 and there was a spike of errors on the REST and GraphQL APIs. Customers would have seen some errors in the Buildkite UI during this period as well. We have been monitoring the situation since then and things have returned to baseline.
Event timeline
How this incident unfolded, from the official status page.
Investigating
BuildKite
We've spotted that something has gone wrong. We're currently investigating the issue, and will provide an update soon.
Resolved
BuildKite
Additional capacity was added to our redis caches. This triggered a failover between UTC 15:10 - 15:14 and there was a spike of errors on the REST and GraphQL APIs. Customers would have seen some errors in the Buildkite UI during this period as well. We have been monitoring the situation since then and things have returned to baseline.