LovableIssues with Lovable Cloud project creation
Fix is completely rolled out, no more issues creating Lovable Cloud projects.
Resolved
Outages and recoveries from the services you depend on.
40 incidents
Fix is completely rolled out, no more issues creating Lovable Cloud projects.
Resolved
We have identified the cause of the delays and our mitigations have been successful. We are continuing to monitor webhook delivery.
Resolved
From June 26, 2026 at 23:40 UTC through June 28, 2026 at 20:55 UTC, Copilot Cloud Agent was degraded. The agent could fail when reporting progress, replying to pull request comments, or opening pull requests. For affected built-in tool calls, the average error rate was approximately 8%, with hourly error rates peaking around 26%.<br /><br />This was due to a regression introduced during a Copilot Cloud Agent runtime deployment that caused several built-in agent tools to become unavailable. In many cases, the affected tool calls failed silently so agent jobs appeared to succeed. This monitoring gap meant it took longer than expected to identify the failure. We mitigated the incident by reverting the runtime deployment to the previously stable version.<br /><br />We've added monitoring and alerting for this class of tool-availability error to reduce time-to-detection. We're also adding regression tests for these built-in agent tools, and improving the shipping safety for future runtime rollouts to avoid similar issues.
Resolved
## **Summary** Between June 22, 2026 at 21:36 UTC and June 25, 2026 at 05:12 UTC, Directory Provisioning for AuthKit was interrupted. During this period, directory updates continued to be received and processed by WorkOS, but downstream AuthKit user and organization membership changes were not applied as expected. The issue was caused by an internal data migration being run in production with parameters that unintentionally changed the AuthKit provisioning state for existing customer directories. Once identified, we restored the affected directory provisioning state, backfilled missed user provisioning updates, and deactivated memberships that had not been deactivated during the incident window. No directory events were lost or dropped during this period; however, events received during the incident period were not immediately propagated to AuthKit. Directories created after the migration were unaffected. ## Background Directory Provisioning for AuthKit works in two steps: WorkOS receives directory events from a customer's identity provider, then propagates those events downstream to create or update AuthKit users and organization memberships. That second step only happens if the directory is marked as eligible for downstream provisioning. If that eligibility is disabled, WorkOS continues receiving directory events normally, but the corresponding AuthKit updates are not applied. ## What Happened On June 22, 2026 at 21:36 UTC, an internal data migration was executed through an administrative migration tool. The migration changed the provisioning status for customer directories in a way that unintentionally disabled Directory Provisioning in AuthKit for affected customers. As a result: * New directory users created during the incident window were not consistently provisioned into AuthKit. * Some directory user updates, including attributes and roles, were not propagated to corresponding organization memberships. * Some users who were removed or deactivated in the directory were not fully deactivated in AuthKit during the incident period. This meant that, for the duration of the incident, some users who should have lost access to an organization may have retained it. Based on our review, we found no evidence that this residual access was exercised or misused during the incident window. The incident was reported on June 24, 2026 after customer reports of provisioning failures, approximately 42 hours after the migration ran. We did not have sufficient monitoring in place at the time that was sensitive enough to detect an abnormal drop in volume for Directory Provisioning in AuthKit, which is why detection relied on customer reports rather than internal alerting. Our team investigated the behavior, identified the migration as the cause, and immediately worked to restore affected directory provisioning state. We then performed additional remediation to address downstream effects from the incident window. This included provisioning missed users, refreshing affected membership attributes and roles, and deactivating stale memberships that should no longer have been active. ## Customer Impact & Recommended Actions Customers with Directory Provisioning enabled for AuthKit during the incident window may have experienced delayed user provisioning, attribute/role updates, or deactivation between June 22, 2026 21:36 UTC and June 25, 2026 05:12 UTC. All known gaps from this window have since been backfilled as part of our remediation. If you have reason to believe your AuthKit users or organization memberships were affected in a way not addressed by this remediation, please reach out to our support team so we can investigate your specific environment. ## **Timeline** * **June 22, 2026 21:36 UTC** – An internal data migration unintentionally disabled Directory Provisioning for affected directories. * **June 24, 2026 15:49 UTC** – The incident was reported after customer provisioning failures were identified. * **June 24, 2026 17:23 UTC** – We restored provisioning status for all affected directories, allowing new provisioning activity to resume. * **June 24, 2026 23:30 UTC** – We completed the first remediation pass to propagate provisioning updates missed during the incident period. * **June 25, 2026 02:10 UTC** – We completed remediation to refresh affected role and custom attribute data. * **June 25, 2026 05:12 UTC** – We completed deactivation remediation and considered the incident fixed. ## **Remediation** ### **Immediate Fixes** We took the following steps to restore service and correct affected data: * Restored the provisioning status for affected directories to resume Directory Provisioning in AuthKit. * Backfilled user creations, updates, and deactivations made by the identity provider during the incident window that had not been propagated to AuthKit users and organization memberships. * Removed the unsafe migration path that caused the incident so it could not be run again. ### **Near-Term Improvements** Additionally, we are making the following changes to reduce the likelihood and impact of similar incidents: * **Improved monitoring and detection:** We are adding additional monitoring and alerting for Directory Provisioning in AuthKit behavior, so we can more quickly respond if a similar issue were to occur in the future. * **Harden internal migration tooling:** We are reviewing access and safeguards for internal migration tools, including stronger controls for destructive operations and clearer scoping requirements. ## **Conclusion** Reliable user provisioning and deactivation are critical parts of identity infrastructure, and this incident did not meet the standard we hold ourselves to. We regret the impact this had on affected customers and their users. The incident has been resolved, affected data has been remediated, and we are implementing the controls and monitoring described above to help prevent similar incidents in the future. Please reach out to our team if you have additional questions on the impact to your environments.
Resolved
This incident has been resolved.
Resolved
The intermittent DNS lookup failures between Managed Kubernetes and Managed Database hostnames in the FRA1 region have been resolved for now but please let us know if you see the issue again. Connectivity has remained stable, and all systems are operating normally. If you continue to experience any problems, please open a ticket with our support team. Thank you for your patience, and we apologize for any inconvenience.
Resolved
The issue has been resolved.
Resolved
Edge Function API requests have returned to normal. This issue is now resolved.
Resolved
The email issue affecting B2B Ninja is resolved and quotes are sending normally again. Quotes sent during the outage (June 29, 2026, 10:00 PM UTC – July 1, 2026, 6:00 AM UTC) may not have gone out. They appear in the dashboard, but the customer may see them stuck as "pending" on the storefront. To fix an affected quote: open it, click Edit, then click Send Quote. This re-sends the email and restores the customer's view. Quotes already showing as sent need no action. We apologize for the disruption. If a quote still looks wrong after re-sending, contact support.
Resolved
On June 25, 2026, between 17:33 UTC and 17:55 UTC, our background job service experienced degradation which increased delays to pull requests, repository pushes, Actions workflows, and Webhooks, with delays peaking at 7m. The issue was caused by underlying hypervisor issues and an incoming traffic spike, causing service timeouts which led to a connection storm and continual rebalances. <br /><br />The issue was mitigated by replacing the problem node at 17:49, after which all services saw recovery by 18:07.
Resolved
This incident has been resolved.
Resolved
Between 17:01 and 17:51 UTC, customers experienced delays accessing payment receipts and viewing recent transactions in their transaction history. Some terminals were also unable to send or print receipts during this period. Payment acceptance was not affected — all payments processed normally throughout the incident.
Resolved
It looks like users are not actually getting limited, but are getting a 100% build used warning ahead of time when triggering a build from the eas cli. Use eas-cli@latest for correct messaging.
Resolved
Incident has now been mitigated. CI pipelines are now resuming. Engineers will continue to monitor this recovery.
Resolved
This incident has been resolved.
Resolved
Between 12:05 UTC and 12:39 UTC on July 01, 2026, customers in ap-southeast-1 and sa-east-1 regions may have experienced failures with creating branches or projects, and with starting computes for existing endpoints. Running computes are not affected. The issue has been fully resolved, and all services are operating normally. We apologize for any inconvenience
Resolved
Our Engineering team has confirmed that the issue impacting Droplet resizes using the "Downscale Anytime" option has been fully resolved. Users can now resize their Droplets using the "Downscale Anytime" option without any issues. We appreciate your patience while we worked to resolve this issue. If you continue to experience any problems, please open a support ticket from your account so our team can investigate further.
Resolved
The fix was applied and we see the issue is not reproducing anymore.
Resolved
Data is now processed live and the system are fully operational.
Resolved
We identified an failing database process on the Storage layer, which has now been resolved by our Engineering team.
Resolved
The issue has been resolved. Please contact our support if you continue to experience any related issues.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
A second issue was found which caused sporadic issues to occur. We've identified, and monitored.
Resolved
We have confirmed that these alerts were false positive and no actual degradation of service occurred. We apologize for any confusion this may have caused.
Resolved
Operations have returned to normal and this issue is resolved.
Resolved
This incident has been resolved.
Resolved
The issue has been resolved.
Resolved
This incident has been resolved.
Resolved
We can confirm that the issue was mitigated at 21:14 UTC on June 30, 2026 and the service has resumed normal operation. Customers and partners can view additional details about the incident by logging in to https://community.akamai.com/customers/s/feed/0D5a700001GabcKCAR or reaching out to Akamai Support. We apologize for the impact and thank you for your patience and continued support. We are committed to making continuous improvements to make our systems better and prevent a recurrence of this issue.
Resolved
An upstream regional networking issue impacted access for some Linear users in India. The issue has now been resolved, and our service is operating normally for all regions.
Resolved
We have rolled back the changes we have made in our infrastructure to fix the firehose ingestion. Firehose ingestion should be back now.
Resolved
We've scaled up number of instances and we're no longer seeing an increase in errors.
Resolved
The incident has been resolved. We have informed the owners of potentially affected organizations by email that they may be affected and what actions to take if they see issues.
Resolved
This incident has been resolved.
Resolved
This issue is resolved and performance on GitLab.com has returned to normal.
Resolved
We are still investigating and are seeing we are stable for now. We are internally following up for next steps.
Resolved
Snowflake Engineering has completed the postmortem of this service incident. A detailed Root Cause Analysis \(RCA\) is available on the Snowflake Community site: [https://community.snowflake.com/s/article/INC0159680](https://community.snowflake.com/s/article/INC0159680)
Resolved
We have not observed any increased rates of connection problems on the us-east-1 Supavisor cluster during our monitoring period. We are confident that the incident is resolved.
Resolved
## **Summary** _Between **06:50 UTC and 07:20 UTC on June 26, 2026**, customers experienced intermittent login failures while accessing Traceable environments across multiple US clusters. The incident was traced to an issue with the external authentication provider \(Auth0\), where elevated socket timeouts caused increased login latency and authentication request failures. Login functionality gradually recovered as the upstream issue stabilized, and the incident was resolved after successful login validation across multiple impacted clusters._ ## **Root Cause** _The root cause was an issue with the external authentication provider \(Auth0\), which experienced elevated socket timeouts while processing authentication requests. These upstream timeouts increased login latency and caused intermittent authentication failures across multiple US-region clusters. Since authentication requests depended on the external provider, affected login attempts failed despite Traceable platform services remaining healthy. The incident was resolved once the upstream authentication service recovered and login requests consistently completed successfully._ ## **Impact** _Starting at approximately 06:50 UTC, customers experienced intermittent login failures when accessing Traceable environments across multiple US clusters. The issue affected user authentication, preventing some users from accessing the platform while underlying application services remained operational. Login functionality progressively recovered during the incident, and normal authentication was restored by 07:20 UTC._ ## **Remediation** _The engineering team worked with the external authentication provider while continuously monitoring authentication health across affected clusters. Login functionality was validated through platform metrics and manual verification across representative environments. After confirming consistent authentication success across impacted clusters, the incident was declared resolved._ ## **Action Items** _To prevent such issues going forward, Harness will,_ _Increase authentication resilience: Evaluate improvements to authentication request handling, including timeout tuning, retry strategies where appropriate, and graceful degradation for transient upstream failures._
Resolved