Performance degradation of legacy storage cluster

Resolved

Started

Lasted 3h 39m

View official incident ↗

Updates

  1. Resolved

    The rollback is complete, and our latency readings are back to nominal levels.

  2. Monitoring

    The rollback of the legacy storage cluster is mostly complete, and request latencies have largely normalized.

  3. Investigating

    We are experiencing a performance degradation in one of our legacy storage clusters after a recent version update. This could lead to timeouts when querying trace information. Work is underway to revert the update. Ingestion of trace data is not impacted.