Microsoft outage: delays delivering to Outlook, Hotmail and Office 365
Identified
Minor impact
Started
Updates
Identified
Delivery delays to Microsoft (Outlook.com, Hotmail and Microsoft 365) Microsoft is still having a service incident (Microsoft reference EX1550543). Mail sent to Outlook.com, Hotmail and Microsoft 365 recipients is still being delayed. The problem is on Microsoft's side. Their servers are still turning away most delivery attempts with temporary errors. Our platform is working normally, and delivery to all other mailbox providers is unaffected. As of 04:45 UTC (10:45 PM US Central), about three in four delivery attempts to Microsoft are still being deferred. Microsoft reports partial recovery in some regions but says it has not held, and they are still working on a fix. What this means for you: You don't need to do anything. We keep accepting your mail, queue anything Microsoft defers and retry it automatically. Most mail is still getting through, but more slowly than usual. If you have set a short expiration on your messages, some messages to Microsoft recipients may expire before Microsoft accepts them. Next update: This is our last scheduled update tonight. Our automated monitoring is still watching delivery to Microsoft around the clock. We will post again as soon as we see Microsoft's servers sustaining recovery, or tomorrow morning (US Central).
Identified
Microsoft reports that the changes it applied to restore service are not working as expected, and it is pursuing other ways to add capacity. Delivery to Outlook.com, Hotmail, Live, MSN and Office 365 recipients remains significantly delayed, and about two-thirds of delivery attempts to Microsoft are still being deferred. Microsoft has not given an estimated time for recovery. The incident is tracked as EX1550543, and Microsoft's updates are at https://status.cloud.microsoft/m365/referrer=serviceStatusRedirect. SparkPost's systems are working normally. Messages still waiting for Microsoft remain queued and are being retried automatically. Please don't resend them, because that will create duplicates once they're delivered. The problem has now lasted nearly ten hours, so messages with expiration times of several hours or more may expire before Microsoft accepts them. Each expired message generates a bounce event, and since expired messages are no longer queued, you can decide whether to resend those.
Identified
Delivery to Outlook.com, Hotmail, Live, MSN and Office 365 recipients is improving, but it hasn't fully recovered. Microsoft has restarted affected servers, is adding capacity, and reports that some regions are coming back online. Over the past hour the rate of temporary errors from Microsoft has dropped by about half, and more queued mail is being delivered on retry. Many delivery attempts are still being deferred, so recipients at Microsoft may keep seeing delays for a while. Microsoft is tracking the issue as EX1550543 and posting updates at https://status.cloud.microsoft/m365/referrer=serviceStatusRedirect. SparkPost's systems are working normally. Messages still waiting for Microsoft remain queued and will keep being retried automatically. Please don't resend them, because that will create duplicates once they're delivered. Messages whose expiration time passed during the outage have expired and generated a bounce event. They are no longer queued, so you can decide whether to resend those.
Identified
Microsoft has identified a likely cause of incident EX1550543. A process on its servers that establishes secure connections is becoming overwhelmed, which stops the servers from handling requests. Microsoft has started rolling back a recent network configuration change that coincides with the start of the problem. It is also testing a fix that it plans to apply across all affected servers. Microsoft expects to post its next update by 02:00 UTC, and you can follow it at https://status.cloud.microsoft/m365/referrer=serviceStatusRedirect. Delivery to Outlook.com, Hotmail, Live, MSN and Office 365 recipients is still heavily delayed. More mail is getting through than earlier this evening, but most delivery attempts to Microsoft are still being deferred, and we have not yet seen the effect of Microsoft's changes. SparkPost's systems are working normally. Messages still waiting for Microsoft are queued and retried automatically, so please don't resend them, because that would create duplicates once they are delivered. Because the problem has now lasted more than seven hours, messages with an expiration time of up to several hours may expire before Microsoft accepts them. Each expired message generates a bounce event, and since expired messages are no longer queued, you can decide whether to resend those specific messages.
Identified
We're still monitoring Microsoft incident EX1550543. Microsoft says it has begun bringing affected servers back into service and is seeing limited recovery in some regions. We're seeing a gradual improvement too. More mail is getting through to Outlook.com, Hotmail, Live, MSN and Office 365 recipients than an hour ago, but most delivery attempts to Microsoft are still being deferred. Delays will continue until Microsoft completes its recovery. Microsoft expects to post its next update by 00:30 UTC, and you can follow it at https://status.cloud.microsoft/m365/referrer=serviceStatusRedirect. SparkPost's systems are working normally. Messages still waiting for Microsoft are queued and retried automatically, so please don't resend them. That would create duplicates once they are delivered. Because the problem has now lasted more than six hours, messages with an expiration time of up to several hours may expire before Microsoft accepts them. Each expired message generates a bounce event in your event data. Expired messages are no longer queued, so you can decide whether to resend those specific messages.
Identified
We're still monitoring Microsoft incident EX1550543. Delivery to Outlook.com, Hotmail, Live, MSN and Office 365 recipients is still heavily delayed, and we haven't seen a meaningful improvement. In its latest update, Microsoft said its earlier recovery steps did not bring the expected relief. It has traced the problem to a core process on its servers that is becoming overwhelmed. Microsoft is now isolating part of its infrastructure, restarting it and slowly bringing traffic back to test a recovery approach before applying it more widely. It has not given a time for recovery, and its next update is expected by 23:30 UTC. You can follow it at https://status.cloud.microsoft/m365/referrer=serviceStatusRedirect. SparkPost's systems are working normally. We're still queuing messages to Microsoft and retrying them automatically, and some continue to get through on a retry. Please don't resend affected messages, because that will create duplicates once the queued mail is delivered. Because this has now lasted several hours, messages with an expiration time of up to a few hours may also expire before Microsoft accepts them, not only very short ones. You'll get a bounce event for each expired message.
Identified
We're still monitoring the Microsoft incident (EX1550543). Delivery to Outlook.com, Hotmail, Live, MSN and Office 365 recipients is still heavily delayed, and we haven't seen a meaningful improvement yet. Microsoft says it has found a way to bring a test group of its affected servers back into service and is now rolling that fix out more widely. It is also restarting servers that stop responding. Delivery may improve gradually as that rollout continues, and Microsoft expects to post its next update by 22:30 UTC. You can follow it at https://status.cloud.microsoft/m365/referrer=serviceStatusRedirect. SparkPost is still queuing and retrying affected messages automatically, and some continue to get through on a retry. Please don't resend them. Messages with a short expiration time may still expire before Microsoft accepts them.
Identified
Microsoft has confirmed an incident affecting Exchange Online and Microsoft 365 connectivity in all regions. It’s tracked as EX1550543, and you can follow Microsoft’s updates at https://status.cloud.microsoft/m365/referrer=serviceStatusRedirect. Since about 17:30 UTC on November 12, Microsoft’s mail servers have been answering a large share of delivery attempts with temporary errors. This affects Outlook.com, Hotmail, Live, MSN and Office 365 recipients. The errors include “451 4.7.0 Temporary server error”, “421 4.3.2 The maximum number of concurrent server connections has exceeded a limit” and “451 4.7.700 PFA agent busy”. The problem is on Microsoft’s side and affects senders across the industry. SparkPost’s own systems are working normally. Affected messages are not lost. SparkPost queues them and keeps retrying automatically, and many are already going through on a retry. You’ll see these attempts as delay events in your event data. Recipients at Microsoft may get mail later than usual until Microsoft resolves EX1550543. Delivery to other mailbox providers is not affected. You don’t need to do anything, and please don’t send these messages again. Resending creates duplicates once the queued mail goes through. If you’ve set a short message expiration (for example on one-time passcodes), those messages may expire before Microsoft recovers. We’re watching this closely and will post an update as soon as things change.