Ingestion Lag

Resolved

Major impact

Started

Lasted 14h 29m

View official incident ↗

Updates

  1. Resolved

    Metric ingestion was paused at 20:13, restarted at 22:30 UTC and all apps metrics are caught up and stable as of 2019-09-18 0:00UTC. Operations are back to normal.

  2. Monitoring

    We are recreating some databases indexes which has forced us to fully pause ingestion. Once the indexes are rebuilt metrics will fill in to current while we continue to fix the root cause of the ingestion lag.

  3. Monitoring

    We are experiencing some ingestion lag. We have identified the issue and we are working on processing the backlog. Your charts will continue to catch up as we process the backlog.