Resolved incident

Vault login issues in US resolved

Keeper · Minor · Started

Duration
15m
Severity
Minor
Status
Postmortem

Summary

Minor · Postmortem

A redshift cluster in the AWS region was unhealthy due to high CPU load from a recent backend API change. DevOps was alerted to the issue and worked with the engineering team to revert the change. Engineering will redeploy the update once the software changes have been implemented, and in addition a change is being made to prevent any redshift related issues to affect vault logins in the future \(KA-5630\). Additional scaling is planned on redshift \(SYS-2595\).

View official incident ↗

Event timeline

How this incident unfolded, from the official status page.

  1. Identified

    Keeper

    We are aware of slow vault login issues in the US data center, and the issue will be resolved in approx. 20 minutes.

  2. Resolved

    Keeper

    The issue has been resolved.

  3. Postmortem

    Keeper

    A redshift cluster in the AWS region was unhealthy due to high CPU load from a recent backend API change. DevOps was alerted to the issue and worked with the engineering team to revert the change. Engineering will redeploy the update once the software changes have been implemented, and in addition a change is being made to prevent any redshift related issues to affect vault logins in the future \(KA-5630\). Additional scaling is planned on redshift \(SYS-2595\).