imgixSource Creation Error for Microsoft Azure Sources
This incident has been resolved.
Resolved
Outages and recoveries from the services you depend on.
40 incidents
This incident has been resolved.
Resolved
A fix has been deployed. Thank you for your patience.
Resolved
This incident has been resolved.
Resolved
This incident is now resolved. Thanks for bearing with us!
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
On Saturday March 30th from 04:15 UTC until 11:30 UTC Pusher channels experienced a partial outage and higher than usual latencies on our US2 cluster. The incident was triggered by a single, high volume customer sending requests that was causing an authentication error. These errors are collated and stored in our error logging system, which can be viewed on the Pusher Dashboard. Due to the exceptionally high number of errors, the logging system was overwhelmed, and caused all writing to the log system to backlog on our socket processes. This in turn resulted in the socket processes taking longer to process messages, and to start failing health checks. The failing health checks caused pods to restart, which resulted in connections being dropped. We identified this issue and took steps to mitigate, attempting to switch off non-critical components of the system to alleviate stress on the system. Eventually it was clear that these steps were having little impact and we deployed a hotfix to disable the error logging system. During the deployment of this hotfix the system hit a limit on the amount of registered targets on our cloud provider’s load balancer. This caused the rolling deployment to take much longer than expected, as it waited for old socket processes to be fully drained and terminated. After coordination with our cloud provider we were able to increase this limit, allowing the deployment to complete and the incident was resolved. There were 2 main windows of impact From 4:15 till 5:40 UTC and From 7:15 till 9:15 UTC From 9:15 till 11:30 service was operating normally and we saw a ramp up in connections.
Resolved
This incident has been resolved.
Resolved
This incident is now resolved. All of the affected trials have now been reactivated. Thanks for bearing with us!
Resolved
Between 6:30 AM UTC to 9:50 AM UTC, we experienced failures in accessing some features from the Admin Portal. The issue has been resolved and the service is operating normally.
Resolved
Between 6:30 AM UTC to 9:50 AM UTC, we experienced failures in accessing some features from the Admin Portal. The issue has been resolved and the service is operating normally.
Resolved
This incident has been resolved.
Resolved
This incident is now resolved. Thanks for bearing with us!
Resolved
This incident has been resolved.
Resolved
We have identified an issue that was preventing some newly fetched Origin images from being inserted into our Origin image cache, causing the same images to be re-fetched multiple times for new render requests. The impact time (UTC) was 3/21 20:30 - 3/29 0:50. The issue is resolved.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved
Resolved
All services operational
Resolved
This incident has been resolved.
Resolved
An issue with the QueryChannel endpoint led to some queries returning HTTP 403 response code. This incident has been resolved.
Resolved
This incident has been resolved, after identifying and resolving our frontend application limitation
Resolved
This incident is now resolved. Thanks for your understanding!
Resolved
A fix was applied a while ago, all the services were observed to be stable and fully operational. Incident stands resolved
Resolved
This incident has been resolved.
Resolved
This incident has been resolved.
Resolved
We have identified and resolved a problem with Forge hosted storage, where some paginated queries kept repeating the last page. The incident was detected by our internal monitoring and was resolved quickly after detection by reverting the deployment. Activating changes recently made to the query cursors for paginated queries introduced a bug that impacted some apps. A small number of requests were impacted over a 16-minute window, while the incident lasted. Timeline: - 25/Mar/24 10:08 p.m. UTC - Impact started, when the changes were deployed to production - 25/Mar/24 10:09 p.m. UTC - Incident was detected - 25/Mar/24 10:24 p.m. UTC - Incident was resolved and impact ended The impact of this incident has been completely mitigated and our monitoring tools confirm that query operations are back to the pre-incident behaviour. We have also resolved the underlying bug and deployed the fix to production, completely eliminating the cause of this incident. We apologise for any inconvenience this may have caused to our customers and our developer community.
Resolved
We are glad to inform you that the fix has been implemented and the incident resolved. We will run a problem investigation to understand what happened and how to prevent it from happening again. Thanks for your understanding & sorry for any inconvenience this has caused. REF: INFRA-6404
Resolved
Planned maintenance for MyJFrog portal is done.
Resolved
This incident is now resolved. Thanks for bearing with us!
Resolved
We are glad to inform you that the fix has been implemented and the incident with viewing form results has been resolved. We will run a problem investigation to understand what happened and how to prevent it from happening again. Thanks for your understanding & sorry for any inconvenience this has caused.
Resolved
We have resolved this incident, and Airtable’s services are functioning normally. We’re sorry for the inconvenience to our customers’ operations.
Resolved
This incident has been resolved
Resolved
The issue has been resolved.
Resolved
This incident has been resolved.
Resolved
The underlying Kubernetes node experienced an issue, which has been resolved after scaling down the affected node. Normal operations have now been restored to all affected systems and services.
Resolved
This incident has been resolved.
Resolved
We are glad to inform you that the fix has been implemented and the incident resolved. We will run a problem investigation to understand what happened and how to prevent it from happening again. Thanks for your understanding & sorry for any inconvenience this has caused.
Resolved
This incident has been resolved.
Resolved
This issue has been resolved.
Resolved