TeamViewerDevice groups issue
This incident has been resolved.
Resolved
Outages and recoveries from the services you depend on.
40 incidents
This incident has been resolved.
Resolved
Zuora encountered issues with Payment processing and Payment Method validations due to the issue at BlueSnap mentioned in the link below. https://status.bluesnap.com/incidents/267281 The impact timelines are below. January 16, 2025 · 10:49 PST to January 17, 2025 · 05:14 PST
Resolved
This incident has been resolved.
Resolved
The incident has been resolved and all actions on the logged in website experience are operational.
Resolved
This incident has been resolved.
Resolved
This issue has been resolved, and all the affected accounts have been successfully reactivated. We apologize for any inconvenience this may have caused.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
Our engineers have been closely monitoring the platform and are declaring this incident resolved. Thank you for your patience.
Resolved
This incident has been resolved.
Resolved
The SMS delivery issue with T-mobile has been resolved.
Resolved
This incident has been resolved.
Resolved
We have observed a brief service degradation in the NA2 production environment. The impact is resolved and we continue to monitor.
Resolved
Between 6:15 AM PST and 6:35 AM PST users experienced service disruption with Llama 3.3 70b. We have deployed a fix and the issue is now resolved.
Resolved
We experienced a temporary downtime of our Downloads and redirect service, which impacted the ability of some users to download translations.
Resolved
This incident has been resolved.
Resolved
We are glad to inform you that the fix has been implemented and the incident resolved. If for any reason you are still impacted by this incident please contact support at https://www.typeform.com/help/contact/ We will run a problem investigation to understand what happened and how to prevent it from happening again. Thanks for your understanding & sorry for any inconvenience this has caused.
Resolved
This incident has been resolved.
Resolved
We have resolved the stability issue on our site, and the performance is back to normal. We appreciate your patience.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
The store is fully functioning again, along with all its apps 🙌
Resolved
The in-apps and sessions postbacks are back to normal and the issue is now fully resolved. Thank you again for your patience.
Resolved
A deployment went out this morning at 09:49 PST that prevented a subset of users from accessible stack pages. This change has been rolled back and impacted users should be able to access stack pages again.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
We have resolved this incident, and Airtable’s services are functioning normally. We’re sorry for any inconvenience this has caused.
Resolved
The issue has been fully resolved. All affected components are operating normally again.
Resolved
The system has stabilized and our processing backlog is recovering.
Resolved
Access to our dashboards is fully available, and we are actively monitoring the related services. If you experience any issues, please try logging in again. Thank you for your continued patience.
Resolved
This incident has been resolved.
Resolved
We've made changes to ensure Warp will not install a corrupted update. If affected when updating from an older Warp version, please reinstall.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
The processing of in-app events is back to normal and the issue is now fully resolved. Thank you again for your patience.
Resolved
We have experienced an issue that caused some messages on the platform to be significantly delayed before delivery. Only a small fraction of our customers and an even smaller fraction of messages are affected. There is NO ACTION REQUIRED on your end; the issue is already resolved. Before I go into further details I want to apologize to those that are affected by this. We know you rely on us for reliable webhook delivery, and we don't take this responsibility lightly. We take extensive measures in order to prevent issues from ever happening, though unfortunately this was caused by a rare race condition, which is why both our manual reviews and testing as well as automated testing didn't catch it. Additionally, since rate-limited (throttled) messages are delayed (by definition) they are excluded from our usual delivery latency metrics and alerting (as they should be delayed). So we weren't alerted either. What is rate-limiting (throttling)? While Svix can handle however many messages you send us, your customers' endpoints may not be able to. In order to avoid overloading your customer endpoints, they can define a maximum delivery rate. If they are sent more messages a second than they can handle, Svix will buffer the requests and send it to them at the rate they set. You can read more about it in the rate-limiting (throttling) docs. We recently made a few improvements to the rate-limiting mechanism, and the issue was related to that. Because the issue was related to rate-limiting it only affected endpoints that had rate-limiting enabled (either by you or your customers). In addition to requiring rate-limiting to be set and enabled, it was only triggered when rate limiting was actually hit (e.g. if you had a rate limit of 100 and 50 messages were sent, there was no rate-limiting so the bug couldn't trigger), and when there was backpressure on the rate-limit (so going above the limit was sustained). In addition to that, it also required AWS SQS to suffer from increased latency (happens on occasion). The race condition that was triggered there, made it so that some messages were not immediately scheduled for an additional trigger (after the throttling ends) which was what was causing the delayed invocation. The issue was first introduced when the new rate-limiting mechanism was rolled to production. Following a testing period in staging, we decided it was ready for production use. We then enabled it for 0.1% of the traffic on the 20th of November, ramping up all the way to 10% of the traffic on the 16th of December. We deployed a fix for the core issue on the 16th of December. The fix was for a benign symptom that was caused by the same core issue. At that stage, however, it was not clear messages rate-limiting could cause delays to such an extent so we didn't proactively search for affected messages. This is why alerting affected customers was delayed. We are very proud of our reliability track record, and we know we fell short of it in this scenario. As mentioned above, we have many measures in place to help us prevent issues like these from ever happening, though in this case they were insufficient due to the rare scenario required to trigger this race condition. All of this doesn't change the fact that your service was adversely affected by this, and I'm extremely sorry about that. Please let me know if you have any questions or if there's anything else I can help you with. – Tom
Resolved
This incident has been resolved.
Resolved
We are glad to inform you that the fix has been implemented and the incident resolved. We will run a problem investigation to understand what happened and how to prevent it from happening again. Thanks for your understanding & sorry for any inconvenience this has caused.
Resolved