Resolved incident
Degraded Service
Vimeo · Critical · Started
- Duration
- 3h 47m
- Severity
- Critical
- Status
- Postmortem
Summary
Critical · Postmortem
On June 12, from approximately 2:00 PM to 3:15 PM EDT, a widespread outage at our cloud provider caused most Vimeo services to become temporarily unavailable. During this time, Vimeo users received generic error messages when attempting to access core aspects of the service, including upload, playback, transcoding, API, analytics, and live streaming. Vimeo service was restored when our cloud provider resolved the issue, although residual effects continued until 4:30 PM EDT while our systems worked through large backlogs in the transcoding pipeline. As a result of this outage, our engineering leadership is reviewing our plan for multilevel cloud failure to improve recovery time in the future.
Event timeline
How this incident unfolded, from the official status page.
Investigating
Vimeo
We are currently investigating an issue with degraded service across various parts of Vimeo.
Investigating
Vimeo
We are continuing to investigate this issue.
Investigating
Vimeo
We are continuing to investigate this issue.
Investigating
Vimeo
We are continuing to investigate this issue.
Investigating
Vimeo
We can confirm that vimeo.com, mobile, and playback are broadly affected.
Identified
Vimeo
We're starting to see recovery across the platform.
Monitoring
Vimeo
Our systems continue to recover with most being completely operational. Live Streaming may still encounter issues with archiving once the stream concludes. Additionally, uploads and video processing may be delayed as the system processes the backlog.
Monitoring
Vimeo
New live streams from this point forward should be fully operational while archives for previous streams may require extended recovery time. Upload and video processing continues to make progress though there is still a backlog of work causing some delays.
Resolved
Vimeo
The upload and video processing backlog has been handled. There is still a very small percentage of jobs that need some intervention to retry which will be progressing over the coming hours. Aside from that, all systems are performing within normal parameters.
Postmortem
Vimeo
On June 12, from approximately 2:00 PM to 3:15 PM EDT, a widespread outage at our cloud provider caused most Vimeo services to become temporarily unavailable. During this time, Vimeo users received generic error messages when attempting to access core aspects of the service, including upload, playback, transcoding, API, analytics, and live streaming. Vimeo service was restored when our cloud provider resolved the issue, although residual effects continued until 4:30 PM EDT while our systems worked through large backlogs in the transcoding pipeline. As a result of this outage, our engineering leadership is reviewing our plan for multilevel cloud failure to improve recovery time in the future.