CI/CD Job processing delayDegraded Performance

Updates

1. Resolved

   8 months ago

   Our team has been monitoring for stability and have taken additional steps to help reduce database saturation. This incident is now resolved. More details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21498

2. Monitoring

   8 months ago

   Background processing has recovered and CI wait times are back to normal. Our team is continuing to monitor for stability.

3. Monitoring

   8 months ago

   We're seeing stabilization and recovery but are still monitoring and discussing next steps. More info in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21498

4. Identified

   8 months ago

   Our team has identified the issue as PgBouncer saturation and are looking at mitigation efforts. Users may see small delays in background processing. More information in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21498

5. Investigating

   8 months ago

   No material updates at this time. Our team is still investigating this issue.

6. Investigating

   8 months ago

   Our team is investigating potential delays with background job processing. This could impact CI/CD job processing.