CI pipelines not starting on merge requests.Partial Service Disruption
Resolved
Updates
Resolved
Incident has now been mitigated and runner backlogs are now back to normal. CI pipelines are now resuming. Engineers will continue to monitor this recovery.
Monitoring
Queues and runner backlog are trending back towards normal, while systems continue to work through the remaining backlog. Engineers are continuing to monitor this.
Identified
Our engineers are still investigating the issue but jobs are starting to be processed with a delay. More details can be found here: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22412
Identified
No material updates to report. We are continuing to investigate the current issue and engineers are working to determine remediation steps.
Identified
We have identified that the sidekiq workers processing the jobs are getting deferred. The team is working on identifying the remediation steps.
Investigating
We are currently investigating an issue affecting CI pipelines not starting on merge requests. Our team is actively investigating the root cause.