Live outage feed

Outages and recoveries from the services you depend on.

Earlier incidents

40 incidents

Back to latest

Sep 4, 2026

  1. AsanaWe are fully down for a subset of customers. We're investigating the issue, and are rolling back a deployment.

    Incident: An internal system responsible for automatically scaling backend server capacity in one of our compute clusters stopped replacing capacity that had been cycled out during routine maintenance. This caused a gradual reduction in available capacity over several hours. A subsequent deployment was activated with insufficient capacity, causing all new requests for that compute cluster to fail. To mitigate the impact, we reverted to a previous release revision, which still had sufficient capacity. Impact: For approximately 30 minutes, customers whose traffic was handled by this compute cluster saw full downtime; other customers were unaffected. No customer data was lost. Moving forward: We have added additional monitoring to detect this type of capacity-scaling failure much earlier, and have added safeguards to prevent deployments from shifting traffic before sufficient healthy capacity is confirmed. _Our metric considers a weighted average of uptime experienced by users at each data center. The number of minutes of downtime shown reflects this weighted average._

    Resolved

  2. ReplitSome users are unable to publish apps with an external DATABASE_URL error

    We've deployed a fix for the issue causing some projects to fail publishing with a DATABASE_URL error, and publishing should work normally now. If you're seeing missing data in your development database, columns that appear to be gone, or a migration with an unusually high number of drop statements, please contact support (replit.com/help) so we can look into it with you directly. Thanks for your patience while we worked through this.

    Resolved

  3. CircleCImacOS Job Delays

    We've seen no further impact to queue times and everything has returned to normal. We appreciate your patience while we worked to resolve and monitor this issue.

    Resolved

  4. AkamaiCertificate Provisioning System (CPS) Issues and Network Connectivity Issues in Atlanta Region

    We became aware of an issue with a network component related to connectivity in the Atlanta, GA region which impacted the Certificate Provisioning System as well as potential impact to other Edge traffic through the region. The issue lasted between 12:21 UTC and 14:19 UTC on September 4, 2026. We can confirm that the issue is now resolved, and the service has resumed normal operation. Customers and partners can view additional details about the incident by logging in to: or reaching out to Akamai Support. We apologize for the impact and thank you for your patience and continued support. We are committed to making continuous improvements to make our systems better and prevent a recurrence of this issue.

    Resolved

  5. GranolaGranola app services down

    Granola services should now be up and running again - let us know at hey@granola.so: mailto:hey@granola.so if you're still having issues. Any notes that you took during the outage should be safe - you may need to hit retry on any notes to generate them fully. We're really sorry for any disruption.

    Resolved

  6. GoDaddyManaged WordPress Hosting and Web Hosting Degraded

    We investigated an issue affecting our Managed WordPress Hosting and Web Hosting services. During this time, customers experienced intermittent timeouts on the Web Hosting control panel and on websites hosted on Managed WordPress platform. This issue has been resolved.

    Resolved

  7. DaytonaElevated errors on sandbox creation

    This incident has been resolved. Error rates on sandbox creation have returned to normal levels and all services are operating as expected. Thank you for your patience.

    Resolved

  8. Vonage APIIssues impacting Verify API Email channel

    Vonage services have been fully restored following today's service interruption reported by our third-party email provider, which began at 2026-09-04 08:01 UTC. Full resolution has been confirmed effective as of 2026-09-04 13:00 UTC. Thank you for your patience and understanding as we worked to restore services as quickly as possible. If you have any questions, please contact api.support.vonage.com.

    Resolved

  9. Customer.ioReduced Deliverability to Microsoft-Hosted Email Addresses

    Microsoft has resolved the underlying issue, we are seeing deliveries to their services resume as normal. Earlier bounces will be retried accordingly. We appreciate your understanding and please do reach out to support if you need any further assistance.

    Resolved

  10. Trigger.devRuns list/logs degraded

    # Postmortem: Missing task logs and traces Date: 2026-09-03 Duration: ~67 minutes Impact: New logs, spans and trace data for some runs were not stored and cannot be recovered ## Summary Between approximately 14:35 and 15:43 UTC on 2026-09-03, batches of new task events (the logs, spans and span events that make up a run's trace) were rejected by ClickHouse, the database behind the trace view, logs view and event search. Each batch was retried three times over about 1.5 seconds and then dropped. The events in those batches are permanently lost. Run execution, run status and the runs list were unaffected throughout, as were logs and traces that had already been stored. We are sorry for the missing data and for making affected runs harder to understand. ## What customers experienced Runs that were emitting logs or spans during the window may have a trace that is empty or missing spans, depending on which of their batches happened to land. The dashboard did not return errors: reads succeeded with fewer or zero rows, so this appeared as missing content rather than a failure. In a sample of dashboard trace queries during the window, about 41% returned no rows, versus about 2% normally. Separately, the OpenTelemetry ingestion endpoint returned some HTTP 500s and became slow, so tasks emitting a lot of telemetry may have logged export warnings. Runs outside the window were not affected. ## Root cause A release included two ClickHouse schema migrations run back to back on the task events table. The first dropped some unused indexes, which scheduled a background mutation over historical data. The second added a new input-only column. ClickHouse applies alters to a table in order, so the metadata change for the new column waited behind the index-drop mutation on the writer replicas. Our migration runner reported both as successful in under two seconds, because an ALTER returns once the replica it is connected to has accepted it, and our deploy pipeline treats that exit code as "the schema is ready". It does not check that the column exists on every writer replica or attempt a trial insert. The same release changed the task-event writer to include the new column in every insert, with no flag, fallback or separate schema-first release. Once the new code was running against replicas that did not yet serve the column, every task-event insert failed. The flush scheduler retries a failed batch three times and then drops it with no durable replay, which turned a temporary schema mismatch into permanent loss. We cancelled the background mutation and rolled the application back, but the failures stopped when ClickHouse Cloud autoscaled our writer replicas under memory pressure from the mutation, and the new replicas came up with the column visible. The rollback would have stopped the loss roughly twenty minutes after it was merged had that not happened. ## What we've done - Bounded the queues on the OpenTelemetry collectors that receive telemetry from task runners, in all regions, so a stalled database no longer causes them to run out of memory and return errors. - Re-rolled the release the following morning with the column visible on every replica, which went out cleanly. - Fixed a collector alert for dropped telemetry that had been configured in a way that meant it could never fire. ## Ongoing - Alert ownership and severity: routing alerts to the owner of the in-flight deploy, and making sustained production insert failures page. - A migration gate that checks capability rather than exit code: wait for outstanding mutations on the affected table to finish, confirm required state, and fail-closed. - A durable buffer for task-event batches that exhaust their retries, so a ClickHouse incompatibility or writer outage becomes a delay we can replay rather than a loss, with metrics that count what was actually dropped. - Ingestion lag monitoring: add alerts on how long it has been since a task event last landed in the database, and on how long merges and schema operations have been outstanding, so stalled or silently failing ingestion pages us regardless of the cause. We treated "the migration ran" as "the database is ready". The changes above are about making the release process prove that before a change goes out. We apologise for the missing logs and traces.

    Resolved

Older incidents