Intermittent Git and CI/CD pipeline failuresDegraded Performance

Resolved

Started

Lasted 11h 49m

View official incident ↗

Updates

  1. Resolved

    After a period of monitoring we have not seen a recurrence of the issue and consider it resolved.

  2. Monitoring

    After rollback of the feature flag we've observed a low error rate for this issue, but are continuing to monitor to ensure it remains that way.

  3. Identified

    We're rolling back the feature flag and exploring alternative options for a long-term fix. We've observed that the scope of impact is limited to a small set of customers. Details in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21689.

  4. Identified

    We have begun rolling out a feature flag to address the intermittent Git clone failures and timeouts within pipelines. More details in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21689.

  5. Investigating

    We are investigating reports of Git clone failures and CI pipeline timeouts. Affected users may see "GitLab is currently unable to handle this request due to load" or "CI configuration fetch from Gitaly timed out." We are actively investigating and will provide updates as we learn more.