Increased app-not-found errors

Updates

1. Resolved

   3 months ago

   This incident has been resolved.

2. Monitoring

   3 months ago

   A fix has been implemented and we are monitoring the results

3. Identified

   3 months ago

   We’ve deployed an additional mitigation to further reduce Corrosion retry pressure and are seeing improvement; we’re continuing to monitor while remaining affected nodes catch up.

4. Identified

   3 months ago

   We’ve applied a mitigation to reduce the impact from Corrosion batch insertion retries and are continuing to monitor while affected nodes catch up.

5. Identified

   3 months ago

   We have identified the issue as failed insertions in a subset of Corrosion batches. These failures trigger retries, which can cause timeouts for other batches.

6. Investigating

   3 months ago

   We are currently investigating app-not-found errors returned by Machines API calls made shortly after creating new applications.