Span & Transaction ingestion delayed in DE

Resolved

Major impact

Started

Lasted 4h 40m

View official incident ↗

Updates

  1. Resolved

    All backlogs have been processed.

  2. Monitoring

    We continue to make progress on the spans backlog with an estimated completion time of 15 minutes.

  3. Monitoring

    We're continuing to make progress on the span ingestion backlog in US.

  4. Monitoring

    We've recovered all ingestion latency in DE. In US, errors have processed their backlog and we continue to make progress on spans in US.

  5. Monitoring

    The problematic change has been reverted and we're processing backlogs.

  6. Monitoring

    The problematic change has been reverted and we're processing backlogs.

  7. Identified

    We've identified the source of latency and are reverting that change.

  8. Investigating

    We are currently investigating ingestion delays for spans, transactions and alerts in DE and US

  9. Investigating

    We are currently investigating ingestion delays for spans and transactions in DE and US

  10. Investigating

    We are currently investigating ingestion delays for spans and transactions in DE