Degraded Log Ingestion

Resolved

Started

Lasted 1h 50m

View official incident ↗

Updates

  1. Resolved

    Log ingestion has returned to normal

  2. Monitoring

    Our capacity mitigations have taken effect and ingestion has returned to normal. Customers should see logs returning to normal.

  3. Identified

    We continue to increase capacity in targeted areas to mitigate log ingestion degradation. It will take some time for the ingestion backlog to be eliminated.

  4. Identified

    We have implemented increase capacity and logs are returning to normal. It will take some time for the ingestion backlog to be eliminated. Some services, such as realtime, storage, and api-gateway, will have lost logs during the ingestion incident.

  5. Identified

    We have identified the issue with log ingestion. We are testing mitigation options, including bringing up additional capacity. We will continue to update this page with our progress.

  6. Investigating

    We are investigating a problem with log ingestion. Users may experience delay in current log data.