Resolved incident
Asana down in US
Asana · None · Started
- Duration
- 35m
- Severity
- None
- Status
- Postmortem
Summary
None · Postmortem
Incident: Starting at 2025-02-05 21:05 UTC, a configuration change caused a large increase in server logs, overloading logging infrastructure and causing server restarts. This caused cascade failure as other servers were overloaded. The configuration change was reverted, and our application recovered after the servers restarted, with recovery by around 2025-02-05 21:30. Impact: Users saw errors loading our application and loading data within the application. No customer data was lost. Moving forward: We have removed the source of logs which resulted in overload, and are improving monitoring and resilience of the system which triggered failure. We will also be moving to staged configuration rollouts to help reduce or avoid customer impact for similar issues. Our metric considers a weighted average of uptime experienced by users at each data center. The number of minutes of downtime shown reflects this weighted average.
Event timeline
How this incident unfolded, from the official status page.
Investigating
Asana
We are seeing a high rate of errors for the Asana application in the US region.
Investigating
Asana
We are continuing to investigate this issue.
Investigating
Asana
We have made configuration changes and are monitoring for recovery.
Monitoring
Asana
A fix has been implemented and we are monitoring the results.
Resolved
Asana
This incident has been resolved.
Postmortem
Asana
Incident: Starting at 2025-02-05 21:05 UTC, a configuration change caused a large increase in server logs, overloading logging infrastructure and causing server restarts. This caused cascade failure as other servers were overloaded. The configuration change was reverted, and our application recovered after the servers restarted, with recovery by around 2025-02-05 21:30. Impact: Users saw errors loading our application and loading data within the application. No customer data was lost. Moving forward: We have removed the source of logs which resulted in overload, and are improving monitoring and resilience of the system which triggered failure. We will also be moving to staged configuration rollouts to help reduce or avoid customer impact for similar issues. Our metric considers a weighted average of uptime experienced by users at each data center. The number of minutes of downtime shown reflects this weighted average.