Errors and stalled predictions on some models

Resolved

Minor impact

Started

Lasted 12h 21m

View official incident ↗

Updates

  1. Resolved

    This incident has been resolved.

  2. Monitoring

    A fix has been implemented and we are monitoring the results.

  3. Identified

    The issue has been identified and a fix is being implemented. Current impact: predictions on OpenAI models are back to normal. Predictions on other models may still return HTTP 500 errors or stay in a processing state while we continue to work on a fix.

  4. Identified

    The issue has been identified and a fix is being implemented. Current impact: prediction requests for some models may return HTTP 500 errors, and some predictions may stay in a processing state. Customers using the API may also see validation errors on responses.

  5. Investigating

    We are currently investigating this issue. Cloudflare is aware of, and investigating, an issue affecting predictions for some models on Replicate. Customers may see prediction requests return HTTP 500 errors, and some predictions may remain in a processing state without completing. Some API responses are also missing version information, which can cause validation errors in client libraries. We are working to understand the full impact and mitigate this problem.