Resolved incident

Uploading errors in Asset Manager

imgix · Minor · Started

Duration
53m
Severity
Minor
Status
Postmortem

Summary

Minor · Postmortem

# Summary Between 2026-01-09 21:26 UTC and 2026-01-12 14:18 UTC, some asset uploads returned an upload failure error in Asset Manager despite the uploads completing successfully. Additionally, legacy video customers experienced transcoding delays as a result of the incident. # What Went Wrong A production misconfiguration caused a validation step in the Asset Manager pipeline to fail. As a result: * The legacy video encoding process did not initiate for uploaded videos * Assets did not appear in Asset Manager because they failed validation While affected assets were still available at the origin and accessible via the Rendering and Video APIs, they did not appear in the Asset Manager UI and were not available through the legacy Long-Form Video API. # What we will do to prevent this in the future We have done the following things: 1. Added more alerting and patched logging gaps to catch the issue sooner 2. Fix a flaw in our build system that allowed a two-step deploy to get out of sync

View official incident ↗

Event timeline

How this incident unfolded, from the official status page.

  1. Investigating

    imgix

    We are investigating an issue where Asset Manager returns errors for uploaded assets. Affected assets can still be served through the Rendering API but will not be visible in Asset Manager. Customers using the legacy Long Form Video feature may also encounter video delivery errors. We are investigating this issue.

  2. Monitoring

    imgix

    We have identified the issue and deployed a fix that applies to newly uploaded assets. We are continuing to investigate a solution to restore visibility for assets uploaded prior to the fix in Asset Manager.

  3. Resolved

    imgix

    The uploading incident is resolved. We will follow up with affected customers to restore visibility for previously uploaded assets.

  4. Postmortem

    imgix

    # Summary Between 2026-01-09 21:26 UTC and 2026-01-12 14:18 UTC, some asset uploads returned an upload failure error in Asset Manager despite the uploads completing successfully. Additionally, legacy video customers experienced transcoding delays as a result of the incident. # What Went Wrong A production misconfiguration caused a validation step in the Asset Manager pipeline to fail. As a result: * The legacy video encoding process did not initiate for uploaded videos * Assets did not appear in Asset Manager because they failed validation While affected assets were still available at the origin and accessible via the Rendering and Video APIs, they did not appear in the Asset Manager UI and were not available through the legacy Long-Form Video API. # What we will do to prevent this in the future We have done the following things: 1. Added more alerting and patched logging gaps to catch the issue sooner 2. Fix a flaw in our build system that allowed a two-step deploy to get out of sync