Degraded OTEL ingestion in US region

Resolved

Started

Lasted 54m

View official incident ↗

Updates

  1. Resolved

    Completely back to normal

  2. Monitoring

    Backlog is almost completely gone. Ingestion delay is < 1 min.

  3. Identified

    A deployment rollout coincided with a traffic spike, resulting in an unusually large backlog. Adjusting our autoscaling to absorb the backlog faster.

  4. Investigating

    We are observing OTEL ingestion backlog in US region