Major Service Outage - Website, API and Pages
Postmortem
Critical impact
Updates
Postmortem
On `2024-08-14` We experienced an outage affecting the Bitly API from `15:03Z` to `15:10Z` and affecting the Bitly website from `15:03Z` to `15:31Z`. The root cause was a failure in our deployment pipeline which caused the incorrect release to be deployed. This change was immediately detected and rolled back but the failure triggered an unknown dependency loop which required a manual rollback to restore some systems. We have already taken steps to resolve some root causes and are evaluating ways to improve our ability to rollback more quickly in similar situations.
Resolved
This incident has been resolved.
Monitoring
We have implemented a fix for the issue that was affecting our Website, API, and Pages and we are seeing full recovery. We will continue monitoring and report back with any updates.
Investigating
Hi community. We’re currently investigating issues impacting our website and API. We apologize if this is causing trouble for you. We’ll be sure to update you as soon as we know more!