Resolved incident

Issues identified with buildkite-agent registration

BuildKite · Minor · Started

Duration
1h 31m
Severity
Minor
Status
Resolved

Summary

Minor · Resolved

Mitigation strategies worked as expected and now agent registration is back to normal. All systems are stable now.

View official incident ↗

Event timeline

How this incident unfolded, from the official status page.

  1. Investigating

    BuildKite

    We're seeing increased latency and error rates for a subset of our customers. We're currently investigating and will provide status updates as they become available.

  2. Investigating

    BuildKite

    We identified issues affecting certain customers when the buildkite-agent attempts to register with Buildkite. When agents are not able to register, builds may start to back up. We've identified a potential root case and are attempting changes to fix it.

  3. Investigating

    BuildKite

    We have rolled out a few changes to fix the underlying issues and are seeing signs of recovery. We'll keep monitoring until all systems are fully recovered and communicate once we are confident everything is stable again.

  4. Monitoring

    BuildKite

    Recovery is underway; we'll keep monitoring until recovery is complete.

  5. Resolved

    BuildKite

    Mitigation strategies worked as expected and now agent registration is back to normal. All systems are stable now.