BasetenElevated errors for GLM 5.2 on Model APIs. **Dedicated Inference is not affected**
This incident has been resolved.
Resolved
Outages and recoveries from the services you depend on.
40 incidents
This incident has been resolved.
Resolved
From 07:10 July 2 to 01:20 July 3 UTC, we saw elevated errors which affected WeChat Pay payment method transactions. This issue has now been resolved by our partner.
Resolved
Commerce engineers were alerted into increased error rate for subset of stores in US from 2:00 AM UTC to 3:08 AM UTC. The issue has now been resolved.
Resolved
This issue has been resolved following a fix implemented by our upstream provider. Service has returned to normal, and project creation is operating as expected. We appreciate your patience and apologize for any inconvenience caused.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
This incident has now been resolved.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
This incident has now been resolved and the service is now fully operational.
Resolved
Let's Encrypt services have returned back to normal. Cluster operations have now returned to normal.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
Resolved
Resolved
Service Restored.
Resolved
Resolved
Services Restored
Resolved
We've restored functionality to our docs assistant by reverting to a stable model. The assistant is now operating normally. We will share a timeline once we have more details.
Resolved
Snowflake Engineering has completed the postmortem of this service incident. A detailed Root Cause Analysis \(RCA\) is available on the Snowflake Community site: [https://community.snowflake.com/s/article/INC20000047](https://community.snowflake.com/s/article/INC20000047)
Resolved
Snowflake Engineering has completed the postmortem of this service incident. A detailed Root Cause Analysis \(RCA\) is available on the Snowflake Community site: [https://community.snowflake.com/s/article/INC20000034](https://community.snowflake.com/s/article/INC20000034)
Resolved
Snowflake Engineering has completed the postmortem of this service incident. A detailed Root Cause Analysis \(RCA\) is available on the Snowflake Community site: [https://community.snowflake.com/s/article/INC20000046](https://community.snowflake.com/s/article/INC20000046)
Resolved
We've confirmed that Lets Encrypt has returned to normal operations. Cluster operations are now back to normal.
Resolved
On June 24, we experienced approximately 30 minutes of severe data ingestion degradation in Honeycomb’s US and EU instances. During this degradation, Honeycomb rejected a significant percentage of inbound telemetry, and customers would have seen up to a half-hour gap in ingested telemetry. Additionally, during this same time window, customers would have experienced data processing degradation and service disruption within Anomaly Detection and Service Maps. A deployment containing a change to how services retrieve dataset schemas into local caches caused a cascading set of failures, resulting in failed remote cache retrievals, and subsequently, a MySQL stampede due to several services falling back to retrieving schemas from the database. This stampede resulted in significant memory utilization growth, causing some services to crash loop with out-of-memory exceptions while retrieving schemas. Customers would have seen 5xx errors from our ingestion APIs when this occurred, because ingestion services were among those that were crash looping. This also included crash looping from services responsible for Anomaly Detection and Service Maps. Right after these service crash loops began, alerts fired for the crashing services, and procedures were run to pin all services back to the last known good build before the breaking change landed. Once services were running the previous deploy’s build, the 5xx error responses from ingestion APIs returned to nominal baseline levels, and services that were crash looping recovered and resumed normal operation. The underlying issue centered on how dataset schema cache payloads and metadata were serialized to memcached for remote cache retrieval, and how memcached clients deserialize that data before writing it to a local cache. The code change contained a feature flag to toggle the serialization behavior, and backwards-compatibility safeguards for clients deserializing different schema versions from memcached. However, services’ schema deserialization prior to the feature flag flip on did not behave as intended, treating each backwards-compatible schema read as a hard error rather than as a fallback, which subsequently caused every remote schema cache read to fall through to the database. Subsequent analysis and review of the code identified the issue and we have re-deployed this change with a fix along with additional tests. Services now correctly and successfully deserialize schemas from memcached in a safe, backwards-compatible manner.
Resolved
The rollout is complete and resolves the compression issues. All listed affected versions should no longer experience any compression-related errors.
Resolved
This incident has been resolved.
Resolved
Issuance has returned to expected performance and we've observed no further impact.
Resolved
This issue is resolved and advisor results are now available in the Dashboard.
Resolved
This issue has now been resolved. Further details can be found here https://support.1e.com/portal/6/CUSTSUP/article/2668429367/Licensing.dll%20certificate%20expired:%20Users%20are%20unable%20to%20login%20to%20the%20Platform%20UI%20and%20the%20coordinator%20service%20will%20not%20stay%20running.
Resolved
Snowflake Engineering has completed the postmortem of this service incident. A detailed Root Cause Analysis \(RCA\) is available on the Snowflake Community site: [https://community.snowflake.com/s/article/inc20000052](https://community.snowflake.com/s/article/inc20000052)
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
On June 30, 2026, between approximately 17:30 UTC and 21:45 UTC, users may have experienced connection timeouts and errors related to the Block Storage service in Singapore Expansion, SP \(sg-sin-2\). The issue began when one host in the cluster was taken down for maintenance while another host unexpectedly encountered network issues. A configuration issue also contributed to the impact. These factors led to a degraded state that affected performance and, to a limited extent, data availability. We mitigated the impact to customers at 21:45 UTC on June 30, 2026 by correcting the network, configuration and cluster issues. We apologize for the impact and appreciate your patience and ongoing support. We are making configuration and operational changes to our systems to help prevent this from happening again, and remain committed to continuous improvement. This summary provides an overview of our current understanding of the incident, given the information available. Our investigation is ongoing, and any information herein is subject to change.
Resolved
This incident has been resolved.
Resolved
Service has been operating normally. This incident remained open due to an automation/workflow oversight and is now being formally closed.
Resolved
This issue has been resolved.
Resolved
The block within the Yahoo infrastructure that resulted in these elevated bounces has been resolved. Customers can safely resend to bounced recipients from this time period. This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
Google Ad conversions are now being sent in realtime, however events are currently missing from the period during which this incident was active. We will be able to replay the failed conversion events though and will reach out to customers directly to notify them that they are impacted.
Resolved