Live outage feed

Outages and recoveries from the services you depend on.

Earlier incidents

40 incidents

Back to latest

Dec 17, 2025

  1. KlaviyoTemplate Cloning Functionality

    As of 8:23 PM US/Eastern on 12/16/2025, this incident has been resolved and template cloning functionality has been fully restored.

    Resolved

Dec 16, 2025

Dec 15, 2025

  1. ConfluenceDegraded performance of Admin Hub, Atlassian Analytics, Confluence Cloud, Focus, Jira, Jira Product Discovery, Jira Service Management, and Rovo

    On December 15th, 2025, Admin Hub, Atlassian Analytics, Confluence Cloud, Ecosystem, Focus, Jira, Jira Product Discovery, Jira Service Management, and Rovo users in the prod-us-east region may have experienced performance degradations and errors on the web page and mobile apps. The issue has now been resolved, and the service is operating normally for all affected customers.

    Resolved

  2. JiraDegraded performance of Admin Hub, Atlassian Analytics, Confluence Cloud, Focus, Jira, Jira Product Discovery, Jira Service Management, and Rovo

    On December 15th, 2025, Admin Hub, Atlassian Analytics, Confluence Cloud, Ecosystem, Focus, Jira, Jira Product Discovery, Jira Service Management, and Rovo users in the prod-us-east region may have experienced performance degradations and errors on the web page and mobile apps. The issue has now been resolved, and the service is operating normally for all affected customers.

    Resolved

  3. FigmaFigma Make Service Disruption

    From 06:30 UTC to 17:20 UTC, Figma Make was experiencing intermittent errors building Make files. A fix has been implemented in Figma Make and this incident is now resolved.

    Resolved

  4. ContentfulIssues with customers logging into Contentful accounts

    Between 08:40 and 09:20 UTC customers experienced errors when trying to log in to Contentful ("Something went wrong"). This was due to a change we made to our bot protection logic. Once we saw the impact of the change, we reverted and the issue was resolved. If any customers continue to see the error after that time period, a refresh will resolve the issue.

    Resolved

Dec 13, 2025

Dec 12, 2025

  1. Datadog US1Delayed APM metric ingestion

    All impact related to APM metrics has been resolved. A separate incident has been created to track the remaining impact in live process data.

    Resolved

  2. UpstashQStash - Message delays for Flow Control configurations

    We identified and fixed a bug that could cause messages with Flow Control enabled to be delayed longer than their configured delay, resulting in unexpectedly long pending times. The fix is in place and the issue should not recur. If you’re still seeing unusually long-delayed messages, please contact support@upstash.com and we can help with remediation.

    Resolved

Dec 11, 2025

  1. Customer.ioUS infrastructure delays with a subset of deliveries sending

    **Duration:** 2h 14m, Nov 24 8:08 AM UTC - 10:22 AM UTC\] **Severity:** P2 **Impact:** Reduced message processing rates across US infrastructure ### What Happened On November 24th, our message rendering service experienced degraded performance during a period of high traffic. The service, which renders your messages for delivery, encountered memory constraints that caused intermittent service restarts and slower processing rates for priority queues across our US infrastructure. ### Customer Impact * **Affected Services:** All message types including campaigns, transactional messages, and journey workflows **Date:** November 24, 2025 * **Geographic Scope:** US-based workspaces * **Performance Impact:** Message processing slowed to approximately 50% of normal capacity during peak impact * **Duration:** 2 hours 14 mins of degraded performance * **Data Integrity:** No messages were lost; all queued messages were successfully processed ### Root Cause Our message rendering service runs on an auto-scaling infrastructure that automatically adjusts capacity based on workload. During this incident, sudden traffic spikes caused individual servers to consume memory faster than our auto-scaling could compensate. When servers reached memory limits, they restarted automatically \(as designed for resilience\), but these rolling restarts reduced our overall processing capacity during a time of peak demand, creating a compound effect. ### Resolution **Immediate fix:** We deployed updated code to our rendering service that better manages memory consumption during traffic bursts, preventing the cascade of restarts that degraded performance. **Why this works:** The update implements more efficient memory allocation patterns and adds throttling mechanisms that prevent any single traffic burst from overwhelming individual servers, regardless of auto-scaling speed. ### What We're Doing to Prevent This 1. **Smarter Resource Management** \(Completed\) * Deployed code optimizations that prevent memory exhaustion during traffic spikes * Implemented per-node workload throttling to maintain stability 2. **Improved Auto-scaling** \(In Progress - Q1 2026\) * Tuning our auto-scaling to be more predictive rather than reactive * Increasing baseline capacity to handle larger bursts without scaling delays 3. **Better Early Warning** \(In Progress\) * Adding memory pressure alerts that trigger before critical thresholds * Implementing graduated responses to traffic spikes \(pre-scaling based on queue depth trends\) ### Our Commitment While our platform maintained data integrity throughout this incident \(no messages were lost\), we understand that processing delays impact your customer engagement timing. We are actively working to ensure all customers’ messages are processed and delivered as efficiently and reliably as possible. ### Questions? Your Customer Success Manager has details specific to your workspace's impact during this incident. For technical questions or to discuss our infrastructure roadmap, please reach out to your account team.

    Resolved

  2. Customer.ioCustomers may be impacted by data delays

    `Incident Summary ` `On December 8th, 2025, beginning at 17:38 UTC, some customers experienced delays in data processing and message delivery. Normal functionality was fully restored at 18:19 UTC, for a total duration of 41 minutes. No data was lost. ` `A failure during the startup of an internal processing service prevented it from becoming fully operational, leading to reduced throughput and increased retry activity in upstream components. ` `Root Cause ` `During startup, one of our processing services loads information about message queues before beginning normal operation. An unexpected queue state left over from a previous configuration caused the service to encounter an error during this process, leading it to restart repeatedly without successfully completing initialization. ` `Because this service is responsible for handing off work to downstream processors, its unavailability resulted in a drop in throughput and a rise in retry traffic. The elevated retries added load to our underlying data layer and contributed to further delays. ` `Resolution and Recovery ` `Engineers identified the failing service, corrected the underlying queue state, and restored the service to full operation. Once stabilized, normal processing resumed and retry volumes returned to expected levels. The system was monitored to confirm full recovery. ` `Corrective and Preventative Measures ` `To prevent recurrence, the team is improving validation during service startup to better handle unexpected queue conditions, refining deployment procedures to detect stalled services sooner, and enhancing monitoring for repeated restart patterns. These improvements are being incorporated into ongoing reliability work. ` `We apologize for any disruption this caused.`

    Resolved

  3. Help ScoutConnections Issues

    Our team has resolved the issue, and performance is now stable across the board; therefore, we're closing this out. We'll continue to monitor all services over the next few hours. Thank you for your patience while we resolved this!

    Resolved

Older incidents