Pipelines failing with 502 errorPartial Service Disruption

Updates

1. Resolved

   10 months ago

   This incident is now resolved. Our team has confirmed that the rollback fixed the issue and we are no longer seeing 502 Bad Gateway errors.

2. Identified

   10 months ago

   Our team has identified that the 502 Bad Gateway errors were related to a Cloudflare Worker deployment in the HTTP router. A rollback of the deployment is showing a decline in 502 errors for the artifact endpoints. Our team is continuing to monitor.

3. Investigating

   10 months ago

   No material updates at this time. Users may still experience intermittent "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096

4. Investigating

   10 months ago

   No material updates at this time. Users may still experience intermittent "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096

5. Investigating

   10 months ago

   Our team is currently working with Cloudflare to investigate the ongoing 502 errors. Users may still experience intermittent "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096

6. Investigating

   10 months ago

   No material updates at this time. Users may still experience intermittent "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096

7. Investigating

   10 months ago

   Users still experience intermittent "502 Bad Gateway" errors when running pipelines. We are currently investigating previous Cloudflare configuration change. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096

8. Investigating

   10 months ago

   We are continuing to investigate intermittent "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096

9. Investigating

   10 months ago

   We are continuing to investigate possible backend performance bottlenecks and other upstream factors that could be causing the intermittent "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096

10. Investigating

    10 months ago

    We are still investigating the 502 intermittent errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096

11. Investigating

    10 months ago

    We are investigating the rate limiting and related configuration changes is ongoing. Users might still experience intermittent 502 errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096

12. Investigating

    10 months ago

    We are investigating a possible root cause, users still experience intermittent "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096

13. Investigating

    10 months ago

    We are still investigating "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096

14. Investigating

    10 months ago

    We are investigating possible saturation. Users will still experience intermittent "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096

15. Investigating

    10 months ago

    We are still investigating "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096

16. Investigating

    10 months ago

    We are currently investigating pipelines failing with "502 Bad Gateway" errors. More details available at https://gitlab.com/gitlab-com/gl-infra/production/-/issues/21096