Resolved incident

Asana down

Asana · None · Started

Duration
41m
Severity
None
Status
Postmortem

Summary

None · Postmortem

Incident: At 2025-02-06 15:37 UTC, a configuration change caused some of our servers to crash, leading to complete downtime for 20 minutes. We rolled back the configuration, but this event triggered a cascade failure in network routing components; our systems recovered over the next 30 minutes, with full recovery by 2025-02-06 16:30. Impact: Users were unable to use Asana during this event in all regions, with downtime including App, API, and Mobile. No customer data was lost. Moving forward: We have added validation to prevent a repeat of this issue, and are updating our networking components to avoid this form of cascade failure. We will be moving to staged configuration rollouts to identify this class of issue without causing downtime for customer traffic. Our metric considers a weighted average of uptime experienced by users at each data center. The number of minutes of downtime shown reflects this weighted average.

View official incident ↗

Event timeline

How this incident unfolded, from the official status page.

  1. Investigating

    Asana

    We are seeing elevated errors for App and API, and are investigating.

  2. Identified

    Asana

    We have identified a cause and are implemented a fix.

  3. Monitoring

    Asana

    A fix has been implemented and we are monitoring the results.

  4. Resolved

    Asana

    This incident has been resolved.

  5. Postmortem

    Asana

    Incident: At 2025-02-06 15:37 UTC, a configuration change caused some of our servers to crash, leading to complete downtime for 20 minutes. We rolled back the configuration, but this event triggered a cascade failure in network routing components; our systems recovered over the next 30 minutes, with full recovery by 2025-02-06 16:30. Impact: Users were unable to use Asana during this event in all regions, with downtime including App, API, and Mobile. No customer data was lost. Moving forward: We have added validation to prevent a repeat of this issue, and are updating our networking components to avoid this form of cascade failure. We will be moving to staged configuration rollouts to identify this class of issue without causing downtime for customer traffic. Our metric considers a weighted average of uptime experienced by users at each data center. The number of minutes of downtime shown reflects this weighted average.