Resolved incident

Extra Notifications

Runscope · None · Started

Duration
2h 19m
Severity
None
Status
Resolved

Summary

None · Resolved

All systems have returned to normal.

View official incident ↗

Event timeline

How this incident unfolded, from the official status page.

  1. Identified

    Runscope

    We've identified an issue causing notifications to be sent on all test runs regardless of settings. We're rolling back the change causing the issue. The behavior of test steps are not impacted by this incident, only notifications are affected.

  2. Monitoring

    Runscope

    The breaking change has been rolled back. A backlog of email notifications is clearing out so notifications will continue to be sent for a short time. The backlog is causing a short delay for new notifications to be sent. We are monitoring the system closely for any other issues.

  3. Monitoring

    Runscope

    The email notification backlog was overwhelmed by the extraneous notifications. As a result, we have cleared out the backlog of notifications that were queued up between 16:30 and 16:52 UTC to resume sending email notifications without a delay. Non-email notifications were not affected by this action.

  4. Resolved

    Runscope

    All systems have returned to normal.