Resolved incident
Increased errors for new render requests
imgix · Major · Started
- Duration
- 2h 13m
- Severity
- Major
- Status
- Postmortem
Summary
Major · Postmortem
## Summary Between approximately 7:49 AM and 10:57 AM PDT on September 1, 2026, a network fault in a Google Cloud Platform region disrupted connectivity to infrastructure that powers a portion of Imgix's image and video rendering pipeline. This resulted in an average error rate ~2% of total requests served during that window. Previously cached images and videos continued to be served normally throughout the incident. ## What Went Wrong Google Cloud Platform experienced a network-level fault affecting compute infrastructure in one of its US regions. The fault disrupted connectivity between the GCP services that make up our rendering infrastructure, preventing a share of render requests in that region from completing. Google has confirmed the disruption was caused by an incident that occurred during their network maintenance and is continuing to review the incident. We may update this report once we receive more information. Despite having multi-zonal configurations for our services, traffic was impacted across the whole region resulting in the degradation we experienced. As error rates rose and fell in waves throughout the incident, we performed a phased cutover of traffic to a healthy region to reduce impact, and returned traffic to normal once the issue cleared. ## What We Will Do To Prevent This In The Future We are refining our criteria and procedures for cross-regional traffic shifts to improve our responses in future incidents.
Event timeline
How this incident unfolded, from the official status page.
Identified
imgix
We have identified a correlation with an ongoing Google Cloud incident, which reports elevated packet loss and errors for multiple services. We will continue to monitor error rates and provide further updates as Google Cloud shares more information.
Identified
imgix
We are continuing to monitor error rates while investigating potential failover options for this incident.
Identified
imgix
We are pushing out changes to redirect traffic and are seeing improved error rates across the service. We will continue to make updates and monitor the situation.
Identified
imgix
We are observing improved error rates across the service. We will continue to monitor the situation.
Monitoring
imgix
Error rates are back to normal. We will continue to monitor the service.
Resolved
imgix
This incident has been resolved.
Postmortem
imgix
## Summary Between approximately 7:49 AM and 10:57 AM PDT on September 1, 2026, a network fault in a Google Cloud Platform region disrupted connectivity to infrastructure that powers a portion of Imgix's image and video rendering pipeline. This resulted in an average error rate ~2% of total requests served during that window. Previously cached images and videos continued to be served normally throughout the incident. ## What Went Wrong Google Cloud Platform experienced a network-level fault affecting compute infrastructure in one of its US regions. The fault disrupted connectivity between the GCP services that make up our rendering infrastructure, preventing a share of render requests in that region from completing. Google has confirmed the disruption was caused by an incident that occurred during their network maintenance and is continuing to review the incident. We may update this report once we receive more information. Despite having multi-zonal configurations for our services, traffic was impacted across the whole region resulting in the degradation we experienced. As error rates rose and fell in waves throughout the incident, we performed a phased cutover of traffic to a healthy region to reduce impact, and returned traffic to normal once the issue cleared. ## What We Will Do To Prevent This In The Future We are refining our criteria and procedures for cross-regional traffic shifts to improve our responses in future incidents.