Some GitHub jobs failing to connect

Resolved

Started

Lasted 1h 31m

View official incident ↗

Updates

  1. Resolved

    This incident has been resolved.

  2. Monitoring

    All failover infrastructure is live and job service is restored. We will continue to address the underlying cause.

  3. Identified

    AWS are experiencing broad capacity constraints around our preferred instance type, we are activating a fallback to secondary instance type to address.

  4. Identified

    Failover capacity is online and job throughput is increasing.

  5. Identified

    Errors seem to be caused in part by lack of instance capacity in our primary runner instance pool, we are activating a failover instance pool to unblock jobs.

  6. Investigating

    We are investigating reports of some GitHub runners failing to connect to GitHub