Elevated API Errors

Resolved

Minor impact

Started

Lasted 3h 40m

View official incident ↗

Updates

  1. Resolved

    This incident has been resolved.

  2. Monitoring

    We have identified the potential cause as a period of packet loss between us-east-1 and us-east-2 in AWS, two regions where Radar is hosted. More information here: https://health.aws.amazon.com/health/status

  3. Monitoring

    On-call engineers are continuing to work on this issue.

  4. Monitoring

    On-call engineers are continuing to work on this issue.

  5. Monitoring

    On-call engineers are continuing to work on this issue.

  6. Monitoring

    We are continuing to see isolated brief jumps in latencies and errors lasting 1-2 minutes. On-call engineers are continuing to work on this issue.

  7. Monitoring

    A fix has been implemented and we are monitoring the results.

  8. Investigating

    We're experiencing an elevated level of API errors and are currently looking into the issue.