Resolved incident

Spans, logs and metrics ingestion stopped

Sentry · None · Started

Duration
8h 48m
Severity
None
Status
Resolved

Summary

None · Resolved

US ingestion is back to normal. EU ingestion is still delayed, but we are now recovering. We have created a new Statuspage incident for tracking EU, follow that for further updates: https://status.sentry.io/incidents/7tfjf5k3m0g3

View official incident ↗

Event timeline

How this incident unfolded, from the official status page.

  1. Investigating

    Sentry

    We are investigating consumers for spans, logs and metrics are not ingested

  2. Monitoring

    Sentry

    The issue was fixed and we are now monitoring the fix

  3. Monitoring

    Sentry

    Backlog is being consumed and we’re still monitoring recovery

  4. Monitoring

    Sentry

    We're still consuming the backlog and monitoring the situation.

  5. Investigating

    Sentry

    We're continuing to investigate delayed ingestion of spans, logs, and metrics in both our US and EU regions.

  6. Investigating

    Sentry

    We're continuing to investigate delayed ingestion of spans, logs, and metrics in both our US and EU regions.

  7. Investigating

    Sentry

    We're continuing to investigate delayed ingestion of spans, logs, and metrics in both our US and EU regions.

  8. Investigating

    Sentry

    We're continuing to investigate delayed ingestion of spans, logs, and metrics in both our US and EU regions.

  9. Investigating

    Sentry

    We're continuing to investigate delayed ingestion of spans, logs, and metrics in both our US and EU regions.

  10. Monitoring

    Sentry

    We've taken steps to mitigate the issue in the US and we're burning our backlog and expect to be have it fully consumed in about 4 hours. We're continuing to investigate the delayed ingestion in our EU region.

  11. Monitoring

    Sentry

    We're continuing burning our backlog in US and expect to be have it fully consumed in about 1 hour. We're continuing to investigate the delayed ingestion in our EU region.

  12. Monitoring

    Sentry

    We're continuing burning our backlog in US and expect to be have it fully consumed within the next hour. We're continuing to investigate the delayed ingestion in our EU region. EU API and dashboards are also affected by this incident.

  13. Monitoring

    Sentry

    We're still burning the last of our backlog in US and expect to be back to normal the next hour. We're continuing to investigate the delayed ingestion in our EU region. EU API and dashboards are also affected by this incident.

  14. Resolved

    Sentry

    US ingestion is back to normal. EU ingestion is still delayed, but we are now recovering. We have created a new Statuspage incident for tracking EU, follow that for further updates: https://status.sentry.io/incidents/7tfjf5k3m0g3