Degraded ingestion for 20 projects

Resolved

Started

Lasted 6h 55m

View official incident ↗

Updates

  1. Resolved

    All projects ingesting as normal

  2. Monitoring

    Offending workload was isolated, performance should be back to normal for other projects

  3. Identified

    S3 backpressure leads to S3 "please slow down" errors across projects with cm* prefix