Buildkite response to the eMudhra certificate compromise

Event timeline

How this incident unfolded, from the official status page.

1. Investigating

   3 days ago

   BuildKite

   The private keys for eight certificates issued by eMudhra, an external certificate authority, have been compromised. Software that still trusts these certificates may accept fraudulent HTTPS certificates. We’re updating the services that power Buildkite and preparing new CLI and Agent releases, which we expect to ship shortly. We’re also tracking fixes to the libraries and trust stores we depend on and will apply them as they become available. We’ll update this page with the new versions and any action customers need to take. Mozilla has published background on the compromise.

2. Monitoring

   3 days ago

   BuildKite

   We're continuing to monitor the situation, and are making releases to many of our components that run in customer infrastructure. At time of writing, we've released versions that distrust the affected eMudhra certificates for: - Buildkite agent: https://github.com/buildkite/agent: - v4.7.0: https://github.com/buildkite/agent/releases/tag/v4.7.0 - v3.138.2: https://github.com/buildkite/agent/releases/tag/v3.138.2 - Buildkite CLI: https://github.com/buildkite/cli/releases/tag/v3.64.2 - bktec: https://github.com/buildkite/test-engine-client/releases/tag/v3.3.1 We recommend updating to these versions to prevent exposure to the affected certificate providers. We will continue to issue updates for the other binaries we distribute, and are updating any internal services that are affected.

3. Monitoring

   3 days ago

   BuildKite

   We’ve finished updating the services that power Buildkite so they no longer trust the affected eMudhra certificates. No action is needed from customers for Buildkite hosted services. For components that run in customer infrastructure, we still recommend upgrading to the releases listed in our previous update (Buildkite agent v4.7.0 / v3.138.2, Buildkite CLI v3.64.2, and bktec v3.3.1). We’re continuing to release updates for the other binaries we distribute and will post them here as they ship.