Live outage feed

Outages and recoveries from the services you depend on.

Earlier incidents

40 incidents

Back to latest

Jul 16, 2026

  1. PaddleSeller dashboard logins are failing with timeouts

    The incident affecting access to the Paddle seller dashboard sign-in has been fully resolved. All services are now operating normally. We are closely monitoring the platform to ensure continued stability. We apologize for any disruption this may have caused. If you have any questions or concerns, please contact our Customer Support team at sellers@paddle.com.

    Resolved

  2. ClerkIncreased latency on our APIs

    There was a service degradation on our caching layer that affected the latency of our APIs. The caching layer is back to normal and latency dropped to normal levels.

    Resolved

  3. CircleCIIssues with network routing for Mac jobs

    ## Summary From 18:00 UTC on July 13, 2026 to 23:18 UTC on July 14, 2026, some customers running jobs on CircleCI's macOS fleet experienced jobs that hung or failed to progress during GitHub fetch steps. For these customers, GitHub fetch steps which would ordinarily take 1-2 minutes took 20-30 minutes, and timed out. The issue was caused by a capacity and routing problem on the network path between our Mac infrastructure provider and GitHub, and the associated intermittent networking issues slowed down fetches from GitHub. During the entire incident, our team worked directly with our Mac infrastructure provider to identify and reroute traffic away from the affected network paths. We resolved an initial occurrence at 19:40 UTC on July 13. The issue recurred at 21:52 UTC that same day, we reopened the incident, and it was fully resolved by 23:18 UTC on July 14. Our macOS fleet had a similar issue several weeks ago. From 20:18 UTC on June 24, 2026 to 03:09 UTC on June 25, 2026, customers experienced a [similar incident](https://status.circleci.com/incidents/gvysjmkf4ct9) affecting the ability of macOS jobs to reach GitHub. That incident was also caused by a network routing problem in our Mac infrastructure provider. We thank our customers for their patience while we worked through this incident. Please see below for specific actions which CircleCI and our Mac infrastructure provider will be taking. Please reach out to our support team with any questions or concerns. The status pages for this incident can be found [here](https://status.circleci.com/incidents/n1tc2lw9q7l0) and [here](https://status.circleci.com/incidents/7cnl777wp9qz). ## Background CircleCI's macOS jobs are hosted on a third-party Mac infrastructure provider. That provider connects to the broader internet, including services like GitHub, over multiple redundant upstream network paths. When one of those paths is experiencing reduced capacity or a routing problem, jobs that are fetching code or dependencies from a destination along that network path can slow down or hang intermittently, even when CircleCI's own platform, the third-party infrastructure provider, and the destination service are all operating normally. ## What Happened _\(All times UTC\)_ At 18:00 on July 13, some customers running macOS jobs began experiencing intermittent failures and delays fetching code and dependencies from GitHub. We opened an investigation and alerted our customers via our status page at 18:55. By 19:04, our Mac infrastructure provider identified increased latency on one of its network paths and rerouted traffic around it. Fetch times recovered, and we moved the incident to `Monitoring` at 19:29, and to `Resolved` at 19:40. At 20:44 UTC, customers reported that the issue had returned. We reopened the incident and updated our status page to `Investigating` at 21:52. Over the following two hours, we worked closely with our infrastructure provider to gather diagnostic data in an effort to isolate the affected network path. In the meantime, we published guidance recommending that customers increase their `no_output_timeout` setting to 15–20 minutes, to allow more time for GitHub fetches to complete during the intermittent slowdowns, and moved the status page to `Monitoring` at 23:35. After publishing the workaround guidance at 23:35 UTC on July 13, we expected the underlying network issue to improve overnight. It did not. At 13:59 UTC on July 14, we confirmed that customers were still experiencing intermittent failures and moved the status page back to `Investigating`. Our engineering team reproduced the failure directly using our own testing tools, which helped confirm this was a general `git-fetch` issue rather than something specific to any particular build tool, container image, or software update. At 15:35, we updated the status page to share that our infrastructure provider was testing changes to its network configuration to isolate the source of the issue. At 16:31, we updated the status page again to share that we'd identified the cause as reduced inbound bandwidth on a specific route, and that we were working with our provider to shift traffic to a different route. At 16:36, our infrastructure provider identified the affected network paths and shifted traffic away from them, fetch times recovered, and at 16:58, we updated the status page to `Monitoring`. By early afternoon, some customers were again seeing slow fetch times, and we moved the status page back to `Investigating` at 20:18. Our infrastructure provider identified two additional network paths with degraded performance and shifted traffic away from them by 22:07. Fetch times stabilized over the following hour, and we marked the incident `Resolved` at 23:18. In total, customers running Mac jobs may have experienced intermittent job slowdowns or failures for portions of the window between 18:00 UTC on July 13 and 23:18 UTC on July 14. Our infrastructure provider has since reported clean test results across its network and believes the underlying capacity issue originated further upstream, closer to GitHub, rather than within its own network. We are continuing to monitor this closely. ## Future Prevention and Process Improvement We are taking the following steps to prevent a recurrence and improve our response time: **We are building additional automated monitoring for macOS fleet GitHub-fetch performance.** Our extensive automated monitoring did not catch this incident. We are now adding end-to-end monitoring for GitHub fetches in particular, in order to detect similar issues proactively. **We are working directly with our Mac infrastructure provider on faster, more proactive detection.** We are asking our provider to build monitoring that can detect a degraded network path and reroute around it automatically, rather than relying on CircleCI to identify and request a reroute during an active incident. **We are turning the diagnostic tooling we built during this incident into a standing capability.** This will let us detect and reproduce this class of network failure on demand going forward, rather than assembling test infrastructure during an active incident. **We are reviewing our incident response process for a sufficient confirmation window before marking a network-related incident Resolved.** This incident briefly recurred after an early, premature resolution; we are formalizing a minimum period of confirmed-clean monitoring before closing incidents of this type. We will continue to provide regular updates along the way as investigation and remediation continues. **We are actively moving from a single provider for Mac infrastructure** **to multiple providers.** This will allow CircleCI to route incoming customer workloads to the most available and best-performing provider. Customer experience is our top priority, and we commit to continually improving the reliability of our systems to match the trust that our customers place in us. Please reach out to our support team with any questions or concerns.

    Resolved

  4. LinodeConnectivity Issue - US-MIA (Miami

    Our team investigated an issue that affected connectivity in our US-MIA (Miami) data center between 21:20 UTC and approximately 23:28 UTC on July 15, 2026. During this window, users may have experienced degraded networking performance and packet loss for Compute services deployed in this region. The issue was resolved after we implemented a fix. We are continuing to work with our third-party service provider to confirm the root cause, as initial evidence points to a campus cross-connect (dark fiber) outage on their infrastructure.

    Resolved

Jul 15, 2026

  1. NeonStuck Operations in EU-Central-1

    Between 21:59 UTC and 22:33 UTC July 15 2026, customers in EU-Central-1 may have encountered failures waking up databases due to certain compute and suspend operations being stuck. This issue is Resolved.

    Resolved

  2. DigitalOceanServerless Inference - Gemma4 Latency Issues Causing Timeouts & Slow Responses

    The deployed fix has successfully restored full functionality, and our monitoring shows that system performance has completely stabilized. Response times for all Gemma 4 inference workflows have returned to normal baseline levels. We will continue to track platform stability moving forward to ensure long-term reliability. We apologize for any disruption this may have caused to your workflows and appreciate your patience throughout the recovery process.

    Resolved

  3. TrueLayerEUR Payments Delayed Settlement

    The backlog of delayed settlement webhooks has now been fully cleared and settlement notifications are back to being processed in real time. This is now resolved. Impact start time: 15/07/2026, 08:00 UTC Impact end time: 15/07/2026, 19:28 UTC Client actions: None

    Resolved

  4. GitHubActions runs are experiencing failures to start

    On July 13, 2026, between 13:11 and 13:53 UTC, some customers experienced failures starting and running GitHub Actions workflows, which also affected Copilot cloud agent sessions and GitHub Pages builds since they depend on Actions. During the peak of the incident, 30% of Actions jobs failed to start and 2% were delayed more than 5 minutes. <br /><br />The incident was triggered by a configuration change in an internal autoscaling component that contained outdated capacity threshold values. This caused a critical Actions service to scale below its required baseline, reducing capacity for workflow processing. We identified the regression, rolled back the change, and restored service capacity. New workflow executions recovered by 13:39 UTC. Full recovery was reached by 13:53 UTC after the queued backlog was drained. <br /><br />To prevent recurrence, we have added deployment guardrails to validate that autoscaling inputs are current and to detect drift between planned and live scaling state before autoscaling changes are applied.

    Resolved

  5. Intercom - USFin slowness

    Fin is back to normal. Between 14:45 - 15:30 UTC, Fin was slower than usual in the US hosting region.

    Resolved

  6. AxiomElevated Queries error rate

    A revert has happened, the query service is now operating normally. The root cause has also been identified and we are actively working on it so queries are not affected again.

    Resolved

  7. BrowserStackAutomate is facing issues

    Between 12:47 and 19:00 UTC, a subset of users running test sessions on Chrome 149 and 150 experienced an elevated rate of idle timeouts. This was caused by an configuration change, which has since been identified and rolled back. The issue is now fully resolved, and sessions on Chrome 149/150 are operating normally.

    Resolved

  8. StripeElevated 3DS errors on Mastercard transactions

    From 2026-07-15 07:00:00 UTC to 09:06:00 UTC, we experienced elevated 3D Secure errors on Mastercard transactions due to an expired certificate on the upstream Mastercard service. Mastercard identified and resolved the issue on their end. We have verified that all certificates are now valid and transactions are processing normally.

    Resolved

  9. ConvexNew deployments unable to push Node Actions to Convex

    This incident has been resolved. We deployed a fix for the recent backend change that was causing pushes containing Node Actions to fail, and we've confirmed that pushes have returned to normal across all deployments. Pushes without Node Actions were unaffected throughout. No action is needed on your end — retrying any previously failed push will now succeed. We apologize for the disruption.

    Resolved

  10. Webflow[RESOLVED] Reports of issues logging in to Webflow

    This incident is now resolved. Some customers experienced issues logging in to Webflow between 23:30 UTC on July 14 and 00:30 UTC on July 15. We’re no longer seeing customer impact related to this issue. Thank you for your patience while we worked to resolve this.

    Resolved

Older incidents