AblyElevated channel attach failure rates in ap-southeast-2
This incident has been resolved.
Resolved
Outages and recoveries from the services you depend on.
40 incidents
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
Our engineers have fixed the issue and Asana which cause global downtime for ~10 minutes and connectivity issues for a small subset of users for around an hour after that. Asana is now running normally for all our customers.
Resolved
On October 15th, we identified that some boards were performing slowly for Workspace Guests. A fix has been released to all affected users, and the slow performance issue is fixed. No further impact has been observed.
Resolved
This incident has been resolved.
Resolved
The issue has been fully resolved. All affected components are operating normally again.
Resolved
This incident has been resolved.
Resolved
**Incident:** On 2024-10-10, between 01:23 and 02:31 UTC a software change caused overload to a shared resource, leading to application errors. The change was rolled back, with recovery for most users around 02:31 and full recovery by 02:39 UTC. **Impact:** Users were unable to start new sessions on the web and desktop apps, though mobile and existing sessions were unaffected. No data was lost. **Next Steps:** We will conduct a retrospective using the 5 Whys method to improve monitoring, tooling, and playbooks. We will also update our software to manage load more efficiently and reduce reliance on globally shared resources to prevent similar issues in the future. Our metric considers a weighted average of uptime experienced by users at each data center. The number of minutes of downtime shown reflects this weighted average.
Resolved
The fix for this has been fully rolled-out.
Resolved
The issue has been fully resolved. Logging in and signing up with Microsoft has been enabled again.
Resolved
This incident has been resolved.
Resolved
This incident is resolved.
Resolved
This incident has been resolved. The queue has been cleared. If you are facing any issues, please reach out to Support.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
SambaNova Cloud endpoints (https://cloud.sambanova.ai/ and https://api.sambanova.ai/) experienced a brief service disruption and are now operational. This disruption took place starting at 3:30 PM PST and ended at 3:42 PM PST on Monday, October 1st, 2024. During this disruption, users were unable to generate queries.
Resolved
This incident has been resolved.
Resolved
This incident has now been resolved.
Resolved
This incident has been resolved.
Resolved
We are glad to inform you that the fix has been implemented and the incident resolved. Users will be able to see responses under the Responses tab and the response counter has been restored. We will run a problem investigation to understand what happened and how to prevent it from happening again. Thanks for your understanding & sorry for any inconvenience this has caused.
Resolved
Emails to Microsoft destinations are now back to normal. Please contact Support if you are still seeing any issues delivering email.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
We are glad to inform you that the fix has been implemented and the incident resolved. We will run a problem investigation to understand what happened and how to prevent it from happening again. Thanks for your understanding & sorry for any inconvenience this has caused.
Resolved
We are glad to inform you that the fix has been implemented and the incident with the Publish button is resolved. We will run a problem investigation to understand what happened and how to prevent it from happening again. Thanks for your understanding & sorry for any inconvenience this has caused.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved. If you continue to see issues, please contact Datadog technical support.
Resolved
The issue has been fully resolved. All affected components are operating normally again.
Resolved
For most of the production cluster, between 13:05 and 14:02 UTC on 2024-10-01, a small proportion of SSE connections were disconnected 30 seconds after establishing a connection. For the duration of that 30s they still worked fine, and connection resumption was working correctly, so this should not have caused any missed messages. The net effect would have been a small increase in subscribe latency about once every 30s. Only the SSE endpoint was affected; websocket traffic (including anyone using an Ably SDK) was not. For customers connected to ap-southeast-2 (Sydney), the duration of the effect was much longer, between 10:06 on 2024-09-30 to 14:02 on 2024-10-01.
Resolved
This incident has been resolved.
Resolved
This issue is now resolved
Resolved
The issue is now resolved.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
The issue is confirmed to be fixed.
Resolved