Live outage feed

Outages and recoveries from the services you depend on.

Earlier incidents

40 incidents

Back to latest

Sep 30, 2026

  1. Vonage APIVoice API - WebRTC Service Degradation | APAC Region

    Vonage services have been fully restored following today’s service interruption, which began at 2026-09-30 04:18 UTC. Full resolution has been confirmed effective 04:40 UTC. Thank you for your patience and understanding as we worked to restore services as quickly as possible. If you have any questions, please contact api.support.vonage.com.

    Resolved

  2. AivenCreation of new service nodes disrupted

    We have resolved the underlying cause of the new node creation and new nodes are able to be created without issues. Should any further issues be encountered, please reach out to support via the Aiven console. We apologise for the inconvenience caused by this issue, and commit to analysing and improving our platform to avoid future incidents similar to this one.

    Resolved

  3. BuildKitemacOS hosted agent jobs not starting

    Between 11:45 and 15:00 (GMT+10) on 28 September, macOS hosted agent jobs could not start because of a misconfigured agent image update. This has been fixed and new jobs are running normally. Jobs from that window still waiting in the Scheduled state won't start on their own. Please cancel and retry them.

    Resolved

  4. StripeVisa authorization errors

    Between 00:42 and 02:24 UTC September 30th, we saw lower authorization rates on Visa network token transactions. This issue is now resolved.

    Resolved

  5. BuildKiteDelayed notifications

    Outbound notification latency remains at expected levels. Some customers may have received duplicate notifications before the underlying error was resolved.

    Resolved

  6. CloudflareWARP connectivity in Newark, NJ

    Between 00:45- 01:35 UTC, Cloudflare WARP and Zero Trust users in Newark, NJ, may have experienced connectivity issues or a degraded Internet experience.

    Resolved

  7. AivenIssue with PostgreSQL major version upgrade

    All customer services affected by this issue have been restored, and no further action is needed on your side. It is now safe to proceed with PostgreSQL major version upgrades. We apologise for the disruption this caused.

    Resolved

Sep 29, 2026

  1. ChatGPT / OpenAISupport available via email

    The issue affecting support chat has been resolved. You can now contact our support team through the Help Center as usual. Thank you for your patience while we worked to restore service.

    Resolved

  2. HarnessDashboard aggregator experiencing timeouts across prod2 since 15:04 UTC (Sept 21)

    ## Summary On Sep 21, 2026, customers in Prod 2 experienced degraded and then largely unavailable deployment dashboards for approximately three hours. The issue was caused by a set of analytics queries behind those dashboards reading far more data than they needed, which exhausted the capacity of the database serving them. A rolling restart of the affected service restored dashboards fully, and the platform has been stable since. ## Customer Impact Dashboard pages loaded slowly or timed out between 14:48 and 18:11 UTC. Pipeline execution and deployment recording run on separate infrastructure and continued to operate normally. No data loss or corruption occurred. ## Root Cause The dashboard queries were missing a time-range filter on the column used to split the largest table into weekly segments, so each query read all historical data instead of only the requested period. Correctness was never affected — the dashboards returned the right results; only the amount of data read to produce them was excessive. As data volume grew, the cost of those reads exceeded what the database could serve, and retries of the failing queries added further load, so the condition did not clear until the service was restarted. ## Mitigation * Performed a rolling restart of the affected service, which stopped the retry cycle and allowed the database to complete its outstanding work * Confirmed dashboard latency and database connection usage returned to normal levels ## Next Steps To prevent recurrence, Harness will: * Add the missing time-range filter to the affected dashboard queries so each reads only the requested period * Replace immediate retries with bounded, backing-off retries * Improve detection by adding alerting on database connection usage and per-query execution cost

    Resolved

  3. HarnessDashboard export and download APIs are currently facing issue.

    # 1. Summary Starting on September 19, 2026, some customers began experiencing failures when downloading or exporting dashboards to CSV through our API, which returned an HTTP 500 \(internal server error\). Standard dashboard viewing, API authentication, and downloads initiated directly from the UI were not affected. This issue was isolated specifically to the automated CSV export and data-extraction API path. ‌ # 2. Impact * only some customers were affected , primarily those relying on automated CSV exports or data-extraction pipelines built on our dashboard export API. * Only dashboards containing certain "merged" or multi-query tiles were affected; dashboards built only from simple, single-query tiles were not impacted. This is why the issue affected some customers and not others. * Standard dashboard viewing, login/authentication, and manual UI-based downloads continued to work normally throughout. # 3. Root Cause This issue was caused by an unintended dependency change introduced during a routine update to the base software image used by our Custom Dashboards service. That update caused the service to automatically pick up a newer version of an internal software library used by our dashboard integration. This newer library version was incompatible with the updated runtime environment introduced in the same change, and that incompatibility caused an error specifically when the export process/ # Remediation We pinned the affected internal library to a known-compatible version, resolving the incompatibility. This fix has been tested and verified to restore correct export behavior for all previously affected dashboard tile types. # Preventive Actions 1. **Pin the runtime version** used in our service's deployment image, so future unrelated changes cannot silently upgrade the underlying runtime and reintroduce a similar incompatibility. 2. **Audit and pin dependency versions** across related internal libraries, so an indirect dependency cannot silently update to an incompatible version during a routine rebuild. 3. **Enhance automated test coverage** for the export path that specifically exercises merged/multi-query dashboard tiles, so this class of issue is caught before reaching production in the future.

    Resolved

  4. AkamaiAkamai Control Center and Configuration Deployment Issues

    We can confirm that the issue was mitigated at 20:25 UTC on September 29, 2026, and the service has resumed normal operation. Customers and partners can view additional details about the incident by logging in to: https://community.akamai.com/customers/s/feed/0D5a700001WM3dyCAD or reaching out to Akamai Support. We apologize for the impact and thank you for your patience and continued support. We are committed to making continuous improvements to make our systems better and prevent a recurrence of this issue.

    Resolved

  5. CloudflareOTP Deliverability Errors

    Cloudflare engineering determined that customers didn't actually experience elevated levels of OTP failures. The rise in failures was an isolated incident without widespread impact.

    Resolved

  6. Fly.ioGRU Networking Issues

    We observed networking issues affecting a subset of our hosts in GRU from 19:37 UTC through 19:59 UTC. This affected platform operations requiring leases, private networking, and internet connectivity to some endpoints.

    Resolved

  7. Supabase401 errors due to JWT rejections

    The latest Supabase version has been released. All users impacted by this issue can now upgrade from the dashboard to resolve this issue. Since this new version is released, we will resolve this issue. If you have any questions about upgrading, please refer to our documentation for more information: https://supabase.com/docs/guides/platform/upgrading Thank you again for your patience while we worked on resolving this issue.

    Resolved

Older incidents