Endpoint startup failures for some projects (~100) in ap-southeast-1 for up to 20 minutes
Resolved
Updates
Resolved
The incident has been successfully resolved, and normal operations have been restored to all affected systems and services. The root cause of the incident was a release of new cloud control plane version, which didn't have any issues itself, but triggered pulling new compute images from the registry on all k8s nodes. In some cases it took unexpectedly long -- up to 15 minutes.
Investigating
Identified: (16:09) We have identified the underlying cause of this incident and are actively working to implement a resolution. Our team is committed to resolving this issue as quickly as possible to minimize any further impact.