Resolved incident

Issues with imgix web tools

imgix · None · Started

Duration
< 1m
Severity
None
Status
Postmortem

Summary

None · Postmortem

# Postmortem # **What happened?** On Feb 20, 2024, at 19:00 UTC, Uploads using the \`/api/v1/sources/upload/\` API endpoint experienced slowness and some timeouts during this time.   The issue had been completely resolved by Feb 21, 2024, 20:00 UTC. # **How were customers impacted?** Between Feb 20, 2024, 19:00 UTC, and Feb 21, 2024, 20:00 UTC, some requests to our upload API experienced slow responses, and a subset of requests resulted in timeout errors. Uploads using the \`/api/v1/sources/<source\_id>/upload-sessions/\` endpoints were unaffected; uploads using the Asset Manager UI were unaffected. # **What went wrong during the incident?** Two compounding issues caused the slowdown. The first is a service update creating an issue with our /api/v1/sources/upload/\` API endpoint.  Secondly, a simultaneous and separate slowdown affected the cloud servers responsible for executing the upload actions.  The service update issue increased the load on our upload function, which was already under strain due to the cloud servers experiencing a slowdown. These factors combined to cause the slow and timed-out responses. # **What will imgix do to prevent this in the future?** We have streamlined the upload process so that this cannot happen again.

View official incident ↗

Event timeline

How this incident unfolded, from the official status page.

  1. Postmortem

    imgix

    # Postmortem # **What happened?** On Feb 20, 2024, at 19:00 UTC, Uploads using the \`/api/v1/sources/upload/\` API endpoint experienced slowness and some timeouts during this time.   The issue had been completely resolved by Feb 21, 2024, 20:00 UTC. # **How were customers impacted?** Between Feb 20, 2024, 19:00 UTC, and Feb 21, 2024, 20:00 UTC, some requests to our upload API experienced slow responses, and a subset of requests resulted in timeout errors. Uploads using the \`/api/v1/sources/<source\_id>/upload-sessions/\` endpoints were unaffected; uploads using the Asset Manager UI were unaffected. # **What went wrong during the incident?** Two compounding issues caused the slowdown. The first is a service update creating an issue with our /api/v1/sources/upload/\` API endpoint.  Secondly, a simultaneous and separate slowdown affected the cloud servers responsible for executing the upload actions.  The service update issue increased the load on our upload function, which was already under strain due to the cloud servers experiencing a slowdown. These factors combined to cause the slow and timed-out responses. # **What will imgix do to prevent this in the future?** We have streamlined the upload process so that this cannot happen again.

  2. Resolved

    imgix

    We are investigating issues with imgix administration tools. There are reports of problems with calling our imgix upload API. The rendering service is not affected.