Live outage feed

Outages and recoveries from the services you depend on.

Earlier incidents

40 incidents

Back to latest

Jun 1, 2026

  1. PostHogEmail delivery issues

    This incident has been fully resolved. The Support product's email channel is operating normally again, and we have verified that all affected functionality has been restored

    Resolved

  2. GitHubDisruption with some GitHub services

    On May 26, 2026, between 15:10 UTC and 16:35 UTC the Copilot service was degraded and many models were no longer available for use. On average, the error rate was ~5% and peaked at 11% of requests to the service. This was due to a change that introduced a configuration mismatch in HMAC signing credentials which caused the list of available models to be truncated. This was mitigated by rolling back the change. This rollback was complete by 15:34 UTC though users continued to see impact until cache TTLs expired. <br /><br />We are working to improve our monitoring and error handling to reduce time to detection and better experience for issues like this in the future.

    Resolved

  3. SnykSnyk Code Degradation

    Between 12:03 and 12:04 UTC today (1st June 2026), Snyk experienced a brief period of degradation affecting Snyk Code scans and custom Rule Extension tests. This impacted scans run via the CLI, IDE plugins, CLI uploads, as well as PR Checks. Services have now fully recovered. Please retry any scans or tests that failed during this window.

    Resolved

  4. Intercom - USDegraded Performance Across Fin and Inbox Services

    On June 1, 2026, between approximately 11:08 UTC and 13:24 UTC, a small subset of customers experienced degraded performance across Fin and Inbox. During this period, affected customers may have experienced inbox loading issues, delays in conversation updates, message delivery failures, conversation assignment issues, and disruptions to Fin replies. This issue is now resolved and all services are restored to normal functionality. Please don’t hesitate to reach out to team@intercom.io: mailto:team@intercom.io if you have any further questions or if you’re still seeing any unexpected behaviour

    Resolved

  5. NeonIssue with project or branch operations in eu-central-1

    Between 10:30 UTC and 11:10 UTC June 01, 2026, customers may have experienced failures while creating or deleting branches, or projects, connections to new database created might have been affected. Existing database connectivity remained unaffected. The issue has been fully resolved, and all services are operating normally. We apologize for any inconvenience. If you require further details, please submit a support ticket.

    Resolved

  6. NeonIssue with project or branch operations is us-east-1

    Between 10:22 UTC and 11:32 UTC, June 01, 2026, customers may have experienced failures while creating new projects or branches. The issue has been fully resolved, and all services are operating normally. We apologize for any inconvenience. If you require further details, please submit a support ticket.

    Resolved

  7. PaddleCard payment issues on Paddle Billing and Paddle Classic checkout

    The issue has now been resolved, and our team is monitoring the situation to ensure normal service continues. We apologize for any disruption this may have caused. If you have any questions or concerns, please reach out to us at sellers@paddle.com "sellers@paddle.com": mailto:sellers@paddle.com.

    Resolved

  8. AkamaiReporting Issues affecting URL Traffic and URL Responses reports

    We can confirm that the issue was mitigated and the service has resumed normal operation. Customers and partners can view additional details about the incident by logging in to: https://community.akamai.com/customers/s/feed/0D5a7000018q2TuCAI or reaching out to Akamai Support. We apologize for the impact and thank you for your patience and continued support. We are committed to making continuous improvements to make our systems better and prevent a recurrence of this issue.

    Resolved

  9. BubbleIncreased traffic volume impacting site availability

    We experienced a significant increase in traffic volume to our site, which caused elevated latency and intermittent availability issues for some users. Our traffic protection systems identified and mitigated the unusual traffic pattern, and service availability has since stabilized. We are continuing to monitor the site closely to ensure performance remains normal. We apologize for any disruption this may have caused and will provide another update if we observe any further impact.

    Resolved

  10. CircleCIMac jobs failing to start

    All Mac jobs are running successfully. Our infrastructure provider has confirmed that the DDoS attack has been mitigated and that services have been restored. Thank you for your patience during this incident.

    Resolved

  11. SlackPartial Degradation Impacting Usergroups

    Our investigation has determined that the issue is isolated to a single customer environment and is not impacting the broader service. As a result, this incident will be removed from the status page and we will continue to work directly with the affected customer to address their issue.

    Resolved

May 31, 2026

  1. BuildKiteEmail deliveries are delayed

    We have received reports email deliveries have not been working, affecting signup and invite emails as well as build notification emails. This issue has now been resolved.

    Resolved

  2. LinearDelays affecting event processing in our US region

    We’ve resolved the delays that were affecting updates for some workspaces hosted in our US region, and service should now be operating normally. We hit an unexpected condition in our event processing that caused the system to back up. If you still see stale information in Linear, please refresh and try again. We are working on a fix to prevent future occurrences, and thank you for your patience while we worked on this.

    Resolved

May 30, 2026

  1. LiveKitElevated Reports of Participant Connection Latency and Errors In US East Region

    ## Summary LiveKit Cloud validates incoming API and connection requests via an internal authentication service, accessed over gRPC. The service has been in place since 2022 and is designed for resilience: it runs as multiple redundant instances, has pod-level health monitoring that automatically reaps unhealthy pods, and is backed by in-memory caches so it can survive transient database failures. On 2026-05-28, between 13:55 UTC and 15:45 UTC, a percentage of requests and new connections in our US East region failed or timed out. The root cause was a rare failure mode on a single instance of the authentication service. The instance remained reachable and its TCP connections stayed alive, but it began responding to gRPC requests extremely slowly, in a way that did not trip our existing pod-level health checks or cause gRPC clients to fail over. We sincerely apologize to customers whose traffic was disrupted. We've let you down, and we are taking this very seriously. In addition to the corrective actions outlined below, we are performing a thorough audit of our systems for other failure modes we may not have anticipated. ## Timeline and Impact \(all times in UTC on 2026-05-28\) * **13:55**: Elevated rate of authentication failures observed. * **14:15**: On-call team notified; investigation begins. * **14:46**: Error rate continues to climb; incident escalated to Sev 1. * **15:35**: US East drained; error rate begins to subside. * **16:32**: Root cause identified; faulty instance shut down. * **16:32**: Webhooks backlogged during the incident begin to deliver. Between 2026-05-28 13:55 UTC and 15:35 UTC, a percentage of API requests and new connection attempts in our US East region failed or timed out. This included new participant connections, SIP connection requests, and agent connections originating in US East. The blast radius of this incident was substantially reduced by our local in-process auth cache. Each service maintains an in-memory cache of recently-seen authentication details, which allowed the majority of requests to continue flowing whenever they landed on a server that had already cached the relevant credentials. The failures concentrated on requests that landed on servers without those credentials cached, which disproportionately impacted customers with lower overall traffic to US East. Existing realtime sessions in US East continued to operate, and all traffic in other regions was unaffected. The impact subsided once US East was drained and traffic was rerouted to neighboring regions. There was also a secondary effect for customers who subscribe to outgoing webhooks. During the incident, webhook deliveries that depended on the affected auth path were queued internally while waiting for a valid signing token. Once the faulty instance was removed and the previously stuck operations unblocked, the queued webhooks were delivered in a burst rather than smoothly over time. Customers whose webhook endpoints have limited throughput may have observed a short-lived spike that exceeded their normal load. ## Root Cause The trigger was a rare hardware failure mode on one instance of our authentication service. Typically, when hardware fails, the underlying machine is shut down either by the hypervisor or by Kubernetes \(via failed health checks\), and gRPC clients reconnect to a healthy instance. In this case, the faulty machine did not terminate. It remained reachable and continued to respond, but extremely slowly. The TCP connection to that instance stayed up, so gRPC clients that had pinned their requests onto that connection continued to use it, and pod-level health checks did not fire. As a result, a percentage of services in US East were unable to fail over to healthy instances and saw their auth requests time out. Two additional fallback mechanisms did not engage as fully as we had designed: 1. **Client-driven cross-region failover.** LiveKit clients can automatically fail over from one region to a neighboring healthy region. However, the failover path itself depended on passing the same authentication gate, so it failed for the same reason. 2. **Local in-process auth cache refresh path.** As noted in the Impact section, the in-process cache absorbed a significant share of requests during the incident. However, services always attempt to refresh auth details against the auth service first to ensure they have the most recent data, and fall back to the cache only when that refresh fails or times out. The refresh attempt has a 5-second timeout. Many client SDKs use a default request timeout shorter than 5 seconds and gave up before the cache fallback could engage, even when the relevant credentials were present in the cache. When the impact was first observed, our initial signal pointed to database connection timeouts in US East. Because of that, we did not immediately drain the region. Two reasons informed that decision: 1. only a portion of requests were failing \(some connections were still succeeding\) 2. if the underlying database had truly been the cause, draining US East would have shifted that load onto neighboring regions and risked spreading the impact rather than containing it. After confirming that the database was healthy and that the issue was isolated to US East, we drained the region. Error rates began dropping immediately as traffic moved to neighboring regions. ## Corrective Actions & Prevention This incident exposed a machine failure mode we had not yet designed for. The following changes have been implemented and will be rolled out in the next week: * **Detect and time out slow gRPC connections from internal clients.** Internal gRPC clients will detect connections that are alive but unresponsive, and tear them down so they can re-establish against a healthy instance. * **Improved load balancing between gRPC servers.** Retries from internal clients will spread across alternative instances, so a single slow server cannot black-hole a meaningful share of traffic. * **Reduce inline auth refresh timeout to 1 second.** This allows the local in-process auth cache fallback to engage well within the timeout budget that client SDKs typically use. * **Allow client-driven cross-region failover to function when auth has failed.** Region failover will no longer depend on passing the auth gate, so a degraded auth service in one region cannot prevent clients from reaching a healthy region. We sincerely apologize for the disruption to customers whose traffic was affected. Thank you for your patience, and we welcome any additional feedback from customers who were impacted.

    Resolved

  2. UpstashIntermittent slowness in Vector US-EAST-1 region

    The issue affecting some Upstash Vector indexes in the US-EAST-1 region has been resolved. Our team investigated the incident and identified the conditions that were contributing to elevated memory pressure on the affected servers. We mitigated those conditions by reducing memory utilization on the impacted nodes, rebalancing affected workloads where needed, and increasing available headroom capacity across the region. All affected indexes should now be operating normally, and based on the mitigations applied, we do not expect this issue to recur. We apologize for any inconvenience this may have caused.

    Resolved

  3. ConvexBrief interruption in availability of some Convex deployments

    A configuration update caused an unintended restart of many Convex deployments, resulting in intermittent unavailability from 00:52 to 00:54 UTC in the US East region and from 01:05 to 01:07 UTC in the EU West region. Service has since returned to normal.

    Resolved

May 29, 2026

  1. NotionSubscription Update Issues

    We experienced a brief incident during which some users were unable to update their subscriptions. Our team identified the cause and implemented a fix. Subscription-related functionality is now operating normally.

    Resolved

Older incidents