Increased dispatch latency and error rates

Event timeline

How this incident unfolded, from the official status page.

1. Investigating

   6 months ago

   BuildKite

   We're observing increased error rates and dispatch latency for a subset of our customers. We're currently investigating and will provide status updates as they become available.

2. Identified

   6 months ago

   BuildKite

   The issue has been identified and a fix is being implemented.

3. Monitoring

   6 months ago

   BuildKite

   We have mitigated the issue causing increased Hosted Agents dispatch latency and intermittent timeout errors for a subset of customers. We identified abnormal workload activity that was placing elevated load on a supporting service, and have now blocked that activity and applied additional protections. Service metrics have returned to normal, and we are continuing to monitor closely.

4. Resolved

   6 months ago

   BuildKite

   Previously elevated loads with Hosted Agents dispatch have fully recovered.