Live outage feed

Outages and recoveries from the services you depend on.

Earlier incidents

40 incidents

Back to latest

Mar 25, 2026

  1. GitHubIncident with API Requests

    On March 11, 2026, between 14:25 UTC and 14:34 UTC, the REST API platform was degraded, resulting in increased error rates and request timeouts. REST API 5xx error rates peaked at ~5% during the incident window with two distinct spikes: the first impacting REST services broadly, and the second driven by sustained timeouts on a subset of endpoints. <br /><br />The incident was caused by a performance degradation in our data layer, which resulted in increased query latency across dependent services. Most services recovered quickly after the initial spike, but resource contention caused sustained 5xx errors due to how certain endpoints responded to the degraded state. <br /><br />A fix addressing the behavior that prolonged impact has already been shipped. We are continuing to work to resolve the primary contributing factor of the degradation and to implement safeguards against issues causing cascading impact in the future.

    Resolved

  2. Convexelevated latency for a small subset of convex deployments

    We had elevated latencies on a single database that required us to temporarily shed load while resizing and reparenting the db. This lead to a period of elevated latencies for all instances on that database followed by downtime for a smaller subset of instances while we performed the maintenance operation. This issue should not occur under normal circumstances and was due an underlying database problem that required manual intervention to resolve. We apologize for the impact to those impacted!

    Resolved

  3. DeepgramDegraded API Performance

    Between 15:16 and 15:32 UTC on March 25, 2026, we noticed an increased rate of API requests returning errors, including 403, 404, and 504 status codes. The issue has been identified and resolved.

    Resolved

  4. KnockMessage event delays

    All message event lag has recovered. We are considering this incident resolved.

    Resolved

  5. ZapierPayment failures for some customers

    Earlier, a script error unintentionally detached saved payment methods for some active customers. This caused increased payment errors and prevented some customers from completing checkout, renewing subscriptions, upgrading plans, or paying invoices. We worked with Stripe to reattach impacted customers’ payment methods in bulk, including a final batch for any remaining affected accounts. Payment processing has resumed and our monitoring indicates payment error rates and invoice processing are back to normal. If you’re still having trouble making a payment, please contact Support: https://zapier.com/app/get-help

    Resolved

  6. BuildKiteIncreased Delays with Hosted Agents

    This incident is now resolved. We are no longer seeing further networking issues with Hosted Agents, which affected delays in creating them for Jobs, resolving external traffic and interactions with Cache - affecting Git Mirror Cloning.

    Resolved

  7. GitHubDisruption with Copilot Coding Agent sessions

    On March 19, 2026, between 01:05 UTC and 02:52 UTC, and again on March 20, 2026, between 00:42 UTC and 01:58 UTC, the Copilot Coding Agent service was degraded and users were unable to start new Copilot Agent sessions or view existing ones. During the first incident, the average error rate was ~53% and<br /> peaked at ~93% of requests to the service. During the second incident, the average error rate was ~99%% and peaked at ~100%% of requests with significant retry amplification. Both incidents were caused by the same underlying system authentication issue that prevented the service from connecting to its<br /> backing datastore.<br /> <br /> We mitigated each incident by rotating the affected credentials, which restored connectivity and returned error rates to normal. The mitigation time was 01:24. The second occurrence was due to an incomplete remediation of the first.<br /> <br /> We are implementing automated monitoring for credential lifecycle events and improving operational processes to reduce our time to detection and mitigation of issues like this one in the future.

    Resolved

  8. GitHubIssues with Copilot Coding Agent

    On March 19, 2026, between 01:05 UTC and 02:52 UTC, and again on March 20, 2026, between 00:42 UTC and 01:58 UTC, the Copilot Coding Agent service was degraded and users were unable to start new Copilot Agent sessions or view existing ones. During the first incident, the average error rate was ~53% and<br /> peaked at ~93% of requests to the service. During the second incident, the average error rate was ~99%% and peaked at ~100%% of requests with significant retry amplification. Both incidents were caused by the same underlying system authentication issue that prevented the service from connecting to its<br /> backing datastore.<br /> <br /> We mitigated each incident by rotating the affected credentials, which restored connectivity and returned error rates to normal. The mitigation time was 01:24. The second occurrence was due to an incomplete remediation of the first.<br /> <br /> We are implementing automated monitoring for credential lifecycle events and improving operational processes to reduce our time to detection and mitigation of issues like this one in the future.

    Resolved

  9. StripeElevated iDeal errors

    Between 14:25 - 14:45 UTC there was a disruption with the iDEAL payment method causing increased error rates for iDEAL transactions. The issue was external to Stripe and is now resolved.

    Resolved

  10. GitHubDisruption with Copilot Coding Agent Sessions

    On March 19, 2026, between 01:05 UTC and 02:52 UTC, and again on March 20, 2026, between 00:42 UTC and 01:58 UTC, the Copilot Coding Agent service was degraded and users were unable to start new Copilot Agent sessions or view existing ones. During the first incident, the average error rate was ~53% and<br />peaked at ~93% of requests to the service. During the second incident, the average error rate was ~99%% and peaked at ~100%% of requests with significant retry amplification. Both incidents were caused by the same underlying system authentication issue that prevented the service from connecting to its<br />backing datastore.<br /><br />We mitigated each incident by rotating the affected credentials, which restored connectivity and returned error rates to normal. The mitigation time was 01:24. The second occurrence was due to an incomplete remediation of the first.<br /><br />We are implementing automated monitoring for credential lifecycle events and improving operational processes to reduce our time to detection and mitigation of issues like this one in the future.

    Resolved

  11. TypeformTypeform Builder - Question blocks don't show answer fields

    We are glad to inform you that the fix has been implemented and the incident resolved. We will run a problem investigation to understand what happened and how to prevent it from happening again. Thanks for your understanding & sorry for any inconvenience this has caused.

    Resolved

  12. MakeDatastores not appearing in make UI

    The issue causing Datastore content to disappear from the UI has been fixed. All Datastore contents are now visible across public and enterprise zones, and all platform features are fully operational.

    Resolved

  13. Black Forest LabsEU cluster increased error rates

    Infrastructure maintenance operation in EU2 region caused temporary disruptions to request processing, resulting in elevated error rates for approximately two hours. The issue resolved automatically and all services are operating normally.

    Resolved

  14. ZapierMonitoring a publicly reported LiteLLM package issue

    We’ve completed our review of the publicly reported LiteLLM package issue. Zapier was not affected. Our review confirmed that the LiteLLM versions used in our environments were not among the publicly reported impacted versions, and we found no evidence of unauthorised access or impact on customer data, Zapier systems, or services. We will continue to monitor for any new information and will share updates if anything changes. If you see any further issues, please reach out to our support team at https://zapier.com/app/get-help.

    Resolved

  15. LambdaGlobal Firewall Rulesets Page Not Loading

    Severity Level: Medium The incident has been resolved. The Firewall page is now working as expected. ----------------------------------------------------------------------------- Again, in the event you’re unsure as to whether your service is still impacted by this, please open a support ticket and we’ll get back to you as soon as possible. • https://support.lambdalabs.com/hc/en-us/requests/new To be automatically notified when the status of this incident changes, please click on the “Subscribe to updates” button.

    Resolved

  16. SnowflakeINC0153706

    Snowflake Engineering has completed the postmortem of this service incident. A detailed Root Cause Analysis \(RCA\) is available on the Snowflake Community site: [https://community.snowflake.com/s/article/INC0153706](https://community.snowflake.com/s/article/INC0153706)

    Resolved

Mar 24, 2026

  1. KernelElevated API error rate

    This incident is resolved, with API error rates returning to baseline levels as of 3:21pm PDT.

    Resolved

  2. GitHubDisruption with some GitHub services

    This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available.

    Resolved

Older incidents