Increase in Workflow runs stuck as pending

Resolved

Major impact

Started

Lasted 5h 58m

View official incident ↗

Updates

  1. Resolved

    This incident has been resolved.

  2. Monitoring

    The fix has been fully rolled out and we are monitoring the results.

  3. Identified

    We are currently rolling a fix for the issue. We are monitoring the results.

  4. Identified

    We are still working on a fix for this issue. While we fix the issue, users experiencing pending Workflow runs can trigger an instant rollback to a working deployment created before Jun 24 21:00 UTC to resume operation on new runs.

  5. Identified

    The issue has been identified. We are continuing to work on a fix for this issue.

  6. Identified

    The issue has been identified and a fix is being implemented.

  7. Investigating

    We are currently investigating an issue where some recently deployed Workflow projects are stuck as pending and not delivering queued messages as expected.