We are seeing issues with apps being able to save data

Resolved

Started

Lasted 1h 1m

View official incident ↗

Updates

  1. Resolved

    Data operations have been restored.

  2. Monitoring

    We have landed a fix and are monitoring the situation.

  3. Identified

    We are estimating ~30 minutes until things are fully operational again

  4. Identified

    We've found an issue with our database upgrade procedure that can cause write operations to fail under specific circumstances. We believe this affects a small number of apps on the main cluster. Given that, we are going to abort and reschedule our upgrade operation, which should restore functionality to the impacted apps. We are working on doing this shortly.

  5. Investigating

    We are currently investigating the scope of the incident, but we believe it affects some main cluster applications. We believe it is related to the database maintenance we are performing, and are looking into the root cause