UI unavailable

Resolved

Critical impact

Started

Lasted 1h 2m

View official incident ↗

Updates

  1. Resolved

    Charts should be caught back up - all systems back to normal.

  2. Monitoring

    The UI is available again. There is a 20 minute lag in data. We've begun replaying data ingestion to fill in the gap.

  3. Identified

    InfluxDB hung while removing a significant amount of data from a timeseries database. We're restarting InfluxDB, which should take around 30 minutes. Data ingestion is continuing - charts will be a bit behind as we replay checkins to Influx after it comes back online.

  4. Investigating

    The Scout UI is currently unavailable. We're investing an issue with our backend timeseries storage.