Spans, logs and metrics ingestion stopped

Resolved

None impact

Started

Lasted 8h 48m

View official incident ↗

Updates

  1. Resolved

    US ingestion is back to normal. EU ingestion is still delayed, but we are now recovering. We have created a new Statuspage incident for tracking EU, follow that for further updates: https://status.sentry.io/incidents/7tfjf5k3m0g3

  2. Monitoring

    We're still burning the last of our backlog in US and expect to be back to normal the next hour. We're continuing to investigate the delayed ingestion in our EU region. EU API and dashboards are also affected by this incident.

  3. Monitoring

    We're continuing burning our backlog in US and expect to be have it fully consumed within the next hour. We're continuing to investigate the delayed ingestion in our EU region. EU API and dashboards are also affected by this incident.

  4. Monitoring

    We're continuing burning our backlog in US and expect to be have it fully consumed in about 1 hour. We're continuing to investigate the delayed ingestion in our EU region.

  5. Monitoring

    We've taken steps to mitigate the issue in the US and we're burning our backlog and expect to be have it fully consumed in about 4 hours. We're continuing to investigate the delayed ingestion in our EU region.

  6. Investigating

    We're continuing to investigate delayed ingestion of spans, logs, and metrics in both our US and EU regions.

  7. Investigating

    We're continuing to investigate delayed ingestion of spans, logs, and metrics in both our US and EU regions.

  8. Investigating

    We're continuing to investigate delayed ingestion of spans, logs, and metrics in both our US and EU regions.

  9. Investigating

    We're continuing to investigate delayed ingestion of spans, logs, and metrics in both our US and EU regions.

  10. Investigating

    We're continuing to investigate delayed ingestion of spans, logs, and metrics in both our US and EU regions.

  11. Monitoring

    We're still consuming the backlog and monitoring the situation.

  12. Monitoring

    Backlog is being consumed and we’re still monitoring recovery

  13. Monitoring

    The issue was fixed and we are now monitoring the fix

  14. Investigating

    We are investigating consumers for spans, logs and metrics are not ingested