Live outage feed

Outages and recoveries from the services you depend on.

Earlier incidents

40 incidents

Back to latest

Apr 4, 2026

Apr 3, 2026

  1. GitHubIncident with Copilot

    On April 1, 2026, between 07:29 and 12:41 UTC, some customers experienced elevated 5xx errors and increased latency when using GitHub Copilot features that rely on `/agents/sessions` endpoints (including creating or viewing agent sessions). The issue was caused by resource exhaustion in one of the Copilot backend services handling these requests, in turn, causing timeouts and failed requests. We mitigated the incident by increasing the service’s available compute resources and tuning its runtime concurrency settings. Service health returned to normal and the incident was fully resolved by 12:41 UTC.

    Resolved

  2. GitHubIncident with Pull Requests: High percentage of 500s

    On Monday March 31st, 2026, between 13:53 UTC and 21:23 UTC the Pull Requests service experienced elevated latency and failures. On average, the error rate was 0.15% and peaked at 0.28% of requests to the service. This was due to a change in garbage collection (GC) settings for a Go-based internal service that provides access to Git repository data. The changes caused more frequent GC activity and elevated CPU consumption on a subset of storage nodes, increasing latency and failure rates for some internal API operations.<br /><br />We mitigated the incident by reverting the GC changes. To prevent future incidents and improve time to detection and mitigation, we are instrumenting additional metrics and alerting for GC-related behavior, improving our visibility into other signals that could cause degraded impact of this type, and updating our best practices and standards for garbage collection in Go-based services.<br />

    Resolved

  3. LambdaNon-admin users are unable to launch instances.

    Severity Level: [High] The incident has been resolved. Launches for team members should now work as expected. ----------------------------------------------------------------------------- In the event you’re unsure as to whether your service is impacted by this incident, please open a support ticket. We’ll get back to you as soon as possible. • https://support.lambdalabs.com/hc/en-us/requests/new To be automatically notified when the status of this incident changes, please click on the “Subscribe to updates” button.

    Resolved

  4. GitHubIssues with metered billing report generation

    On March 31, 2026, between 06:15 UTC and 15:30 UTC, the GitHub billing usage reports feature was degraded due to reduced server capacity. Customers requesting billing usage reports and loading the top usage by organization and repository on the billing overview and usage pages were impacted. The average error rate for usage report requests was 15%, peaking at 98% over an eight-minute window. For the billing pages, an average of 56% of requests failed to load the top usage cards. The root cause was an increase in billing usage report requests with large datasets, which exhausted the capacity of the nodes responsible for reporting data. There was no impact on billing charges. <br /><br />We mitigated the incident by adjusting our auto-scaling thresholds to better meet our capacity needs. We are working to improve our metrics to reduce time to detection and mitigation for similar issues in the future.

    Resolved

Apr 2, 2026

  1. GitHubElevated delays in Actions workflow runs and Pull Request status updates

    On March 30, 2026, between 10:11 UTC and 13:25 UTC, GitHub Actions experienced degraded performance. During this time, approximately 2.65% of workflow jobs triggered by pull request events experienced start delays exceeding 5 minutes. The issue was caused by replication lag on an internal database cluster used by Actions, which triggered write throttling in our database protection layer and slowed job queue processing. <br /><br />The replication lag originated from planned maintenance to scale the internal database. Newly added database hosts triggered guardrails in the throttling layer, restricting write throughput. The incident was mitigated by excluding the new hosts from replication delay calculations. <br /><br />To prevent recurrence, we have updated our maintenance procedures to ensure new hosts are excluded from throttling assessments during scaling operations. Additionally, we are investing in automation to streamline this type of maintenance activity.

    Resolved

  2. GhostScheduled posts not being published

    This incident is now resolved. Posts should now publish at the correct scheduled time. Posts that were scheduled to be published between 10:00 UTC and 21:00 UTC on 2 April 2026 may not have been published at the intended time, but they should have been published during the rollout of the fix.

    Resolved

  3. InngestDashboard down

    The incident is now resolved and the system is full operational. Related to the Vercel incident, we updated our dashboard to Node 22.x to solve the issue. We continue to monitor Vercel's incident and react accordingly. https://www.vercel-status.com/incidents/5r9bp5y8rql2

    Resolved

  4. TrueLayerDelayed Settlement for EUR Payments

    You may have experienced delays in EUR payment settlements. This is now resolved and EUR payments are settling at a timely manner. Impact start time: 02/04/2026, 12:32 UTC Impact end time: 02/04/2026, 15:02 UTC Client actions: None

    Resolved

Older incidents