Resolved incident
Subtle changes in how payloads are sent broke signature verification for consumers verifying incorrectly
Svix · None · Started
- Duration
- < 1m
- Severity
- None
- Status
- Resolved
Summary
None · Resolved
We changed the code to send the payload exactly the same way as it's sent to us (before, we were compacting it before sending). This meant that people that relied on the payload to be compact in order to verify webhooks (i.e they were verifying incorrectly) had verification failing. We reverted this immediately once it was reported signatures were failing for customers. While not a bug in Svix, as people verifying webhooks correctly wouldn't have had issues, it still caused disruption to our customers and we've added tests to ensure that we now always compact the payloads going forward. We are also working on making it even harder to get wrong for customers, but please refer to https://docs.svix.com/receiving/verifying-payloads/how for the correct way of verifying webhooks.
Event timeline
How this incident unfolded, from the official status page.
Resolved
Svix
We changed the code to send the payload exactly the same way as it's sent to us (before, we were compacting it before sending). This meant that people that relied on the payload to be compact in order to verify webhooks (i.e they were verifying incorrectly) had verification failing. We reverted this immediately once it was reported signatures were failing for customers. While not a bug in Svix, as people verifying webhooks correctly wouldn't have had issues, it still caused disruption to our customers and we've added tests to ensure that we now always compact the payloads going forward. We are also working on making it even harder to get wrong for customers, but please refer to https://docs.svix.com/receiving/verifying-payloads/how for the correct way of verifying webhooks.