Elevated 401 errors for API endpoints

Resolved

Started

Lasted 36h 1m

View official incident ↗

Updates

  1. Resolved

    All impacted services have now fully recovered.

  2. Monitoring

    We have applied the mitigation and are monitoring the recovery.

  3. Identified

    We've identified that intermittent 401 errors on some API endpoints are being caused by timeouts. We have applied mitigations that reduced impact in parts of the system and are continuing to deploy additional fixes and monitor recovery until error rates return to expected levels.

  4. Identified

    We've identified that intermittent 401 errors on some API endpoints are being caused by timeouts. We have applied mitigations that reduced impact in parts of the system and are continuing to deploy additional fixes and monitor recovery until error rates return to expected levels.

  5. Identified

    We've identified that intermittent 401 errors on some API endpoints are being caused by timeouts. We have applied mitigations that reduced impact in parts of the system and are continuing to deploy additional fixes and monitor recovery until error rates return to expected levels.

  6. Identified

    We have identified that users are experiencing elevated errors for the impacted services. We are working on implementing a mitigation.

  7. Monitoring

    We have applied the mitigation and are monitoring the recovery.

  8. Investigating

    We are investigating the issue for the listed services.

  9. Monitoring

    We have applied the mitigation and are monitoring the recovery.

  10. Investigating

    We are investigating the issue for the listed services.