Resolved incident

Scheduled report functionality is disabled

Opsgenie · Minor · Started

Duration
146h 42m
Severity
Minor
Status
Resolved

Summary

Minor · Resolved

We're excited to inform you that we've shipped upgrades to our production environment, enabling scheduled reports once again. What's Changed: To continuously improve and ensure the security of our services, we've implemented additional controls including domain restrictions and a limitation on the number of recipients per email. This is specifically for mitigation purposes. From now on, users will start receiving emails for the reports they've scheduled for themselves, and they will also have the ability to create new tasks. Impact: Please note that any existing scheduled jobs with external recipients will no longer be editable. However, users can delete these and create new jobs using their email IDs. Thank you for your patience during these changes. We want to assure you that future updates and communications will be shared promptly to keep you informed. We appreciate your understanding and continued support.

View official incident ↗

Event timeline

How this incident unfolded, from the official status page.

  1. Investigating

    Opsgenie

    Scheduled report functionality is disabled as we suspect a possible spam activity. Only the reports with a custom schedules are disabled, periodic emails are not impacted. The team is investigating the issue and will provide more update.

  2. Identified

    Opsgenie

    The cause of the issue is identified, and the team is working on the fix.

  3. Identified

    Opsgenie

    The root cause is identified and the spam activity is mitigated. The team is working on adding more controls to prevent further spam activities. The scheduled report feature will be kept disabled for a while until the further controls implemented. However, the reporting service is fully available and the reports can be downloaded manually via the reporting page.

  4. Identified

    Opsgenie

    We have implemented additional controls and are introducing domain restrictions as well as limitations to the number of recipients for mitigation purposes. Scheduled reports will be enabled for all customers on November 24th, PST. Users will begin receiving emails for the reports they have scheduled for themselves and they will also have the ability to create new tasks. We will share more updates when the changes are fully implemented.

  5. Monitoring

    Opsgenie

    The changes have been shipped to production, and scheduled reports are enabled now.

  6. Resolved

    Opsgenie

    We're excited to inform you that we've shipped upgrades to our production environment, enabling scheduled reports once again. What's Changed: To continuously improve and ensure the security of our services, we've implemented additional controls including domain restrictions and a limitation on the number of recipients per email. This is specifically for mitigation purposes. From now on, users will start receiving emails for the reports they've scheduled for themselves, and they will also have the ability to create new tasks. Impact: Please note that any existing scheduled jobs with external recipients will no longer be editable. However, users can delete these and create new jobs using their email IDs. Thank you for your patience during these changes. We want to assure you that future updates and communications will be shared promptly to keep you informed. We appreciate your understanding and continued support.