Resolved incident
Widespread Access Outage
Klaviyo · Critical · Started
- Duration
- 32h 8m
- Severity
- Critical
- Status
- Resolved
Summary
Critical · Resolved
All services have been restored and are operating normally. Monitoring is complete, and this incident has been resolved. We appreciate your patience throughout this incident.
Event timeline
How this incident unfolded, from the official status page.
Investigating
Klaviyo
Starting around 2:50am EST, due to outages with an upstream provider, several services in Klaviyo including log-in and sending are inaccessible. We'll provide updates as soon as more information is available.
Monitoring
Klaviyo
As of 5:30am EST, the ability to log in and access the Klaviyo app has been restored. Restoration of all functionality and services is in progress. We are actively investigating the state of sending (campaigns, flows). We will continue to monitor services as functionality is repaired and will share updates as we make progress.
Monitoring
Klaviyo
As of 6:15am EST, we are continuing to stabilize our systems and process any actions that were affected by the provider outage, which lasted from approximately 2:50 to 5:30am EST. We are attempting to send any campaigns and flows messages that couldn't be sent during that time. We are also processing events and profile information received during that time. Data from API requests that received an error during the incident were lost. We will continue to share updates as we make progress.
Monitoring
Klaviyo
As of 8:30am EST, we have successfully sent the delayed campaign and flow messages. Over the next several hours, we will process delayed conversion attribution analytics and delayed profile unsubscribes. We continue to closely monitor our systems.
Monitoring
Klaviyo
We continue to process delayed conversion attribution analytics and delayed profile unsubscribes. We are currently experiencing delays in segmentation processing for a subset of users by up to 60 minutes. We’re working to mitigate these processing delays as quickly as possible.
Monitoring
Klaviyo
Segmentation processing has been restored. We continue to work through processing for the delayed analytics and unsubscribes.
Investigating
Klaviyo
We are currently investigating an issue impacting flow sending beginning around 12:00 PM US/Eastern. We’ll provide updates as soon as more information is available.
Investigating
Klaviyo
We are currently experiencing delays in segmentation processing for a subset of users by up to 2.5 hours. We’re working to mitigate these processing delays as quickly as possible.
Identified
Klaviyo
A mitigation has been deployed for the issue impacting flow sending. Any delayed messages are now being replayed and flow performance is improving. We will confirm once this has been fully restored. The team continues to work through the segmentation processing delays. We continue to monitor closely and appreciate your patience throughout this incident.
Monitoring
Klaviyo
Segmentation processing has been restored to real-time. Segments relying on custom object properties remain delayed and we're continuing to restore normal processing for these. We expect flow sending to be fully caught up within the next hour. We continue to process delayed conversion attribution analytics and delayed profile unsubscribes.
Monitoring
Klaviyo
Flow sending is fully caught up and restored to real-time. We continue to process delayed conversion attribution analytics and delayed profile unsubscribes.
Monitoring
Klaviyo
As of 7:32pm EST, we have successfully processed the delayed unsubscribe events and conversion attribution analytics has been restored. We continue to monitor service functionality as repairs have been completed.
Resolved
Klaviyo
All services have been restored and are operating normally. Monitoring is complete, and this incident has been resolved. We appreciate your patience throughout this incident.