Degraded performance for classic Duo chatDegraded Performance

Resolved

Started

Lasted 2h 35m

View official incident ↗

Updates

  1. Resolved

    We observe no more error on Classic Duo chat and confirm successful responses in the UI. For further details, see gitlab.com/gitlab-com/gl-infra/production/-/work_items/21059

  2. Monitoring

    We have deployed the changes to production. We're monitoring the situation. For further details, see gitlab.com/gitlab-com/gl-infra/production/-/work_items/21059

  3. Identified

    We're deploying the revert change to production. For further details, see gitlab.com/gitlab-com/gl-infra/production/-/work_items/21059

  4. Identified

    The cause of degraded performance has been identified. We are preparing a code change to revert the recent deployment that introduced the issue. A merge request has been opened to restore functionality: https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/merge_requests/4226 For further details, see gitlab.com/gitlab-com/gl-infra/production/-/work_items/21059

  5. Investigating

    We are investigating degraded performance with classic Duo chat. Requests may fail intermittently at this time. For further details, see https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21059