Resolved incident
Lever -hosted job site down
Lever · Critical · Started
- Duration
- 1h 59m
- Severity
- Critical
- Status
- Postmortem
Summary
Critical · Postmortem
## **Incident RCA: Job Site Outage** **Incident Date:** October 3, 2025 **Start Time:** 4:51 PM ET **End Time:** 6:00 PM ET ### **Impact Summary** Between 4:51 PM and 6:00 PM ET on October 3, 2025, users experienced degraded performance and eventual unavailability across multiple job boards. ### **Root Cause** The outage was triggered by an unexpected pattern of incoming requests that, while not unusually high in volume, each initiated resource-intensive queries. This combination of request frequency and prolonged query execution overwhelmed backend systems, leading to degraded performance and service unavailability. The traffic resembled automated crawling behavior, which our systems were not optimized to handle at that level of query complexity and concurrency. ### **Resolution** To reduce server load and restore service availability, the impacted job site was temporarily disabled. This action immediately alleviated pressure on backend systems and allowed all job boards to recover. The site remained offline over the weekend and was re-enabled on October 6, 2025, after Engineering confirmed that the irregular traffic had subsided. ### **Preventive Actions** To prevent similar incidents in the future, the following measures have been implemented: * **Enhanced Caching:** Improvements to our job site caching mechanisms now limit the ability of similar request patterns to generate excessive server queries. * **Updated Crawling Guidelines:** We have revised our web crawling directives to encourage automated crawlers to reduce the frequency of their requests. This change helps maintain system stability by minimizing the impact of high-volume automated traffic from compliant sources.
Event timeline
How this incident unfolded, from the official status page.
Identified
Lever
At 2:04pm PDT we identified an issue causing Lever-hosted job sites and Postings API to be down. Users will notice a looping loading wheel and the page may timeout while loading. We are actively investigating the issue to implement a fix. Additional updates will be provided as soon as a resolution is in place.
Identified
Lever
We are continuing to work on a fix for this issue.
Identified
Lever
We have implemented a fix for the issue. All Lever-hosted job site services should be available. We are continuing to monitor.
Monitoring
Lever
A fix has been implemented and we are monitoring the results.
Resolved
Lever
This incident has been resolved.
Postmortem
Lever
## **Incident RCA: Job Site Outage** **Incident Date:** October 3, 2025 **Start Time:** 4:51 PM ET **End Time:** 6:00 PM ET ### **Impact Summary** Between 4:51 PM and 6:00 PM ET on October 3, 2025, users experienced degraded performance and eventual unavailability across multiple job boards. ### **Root Cause** The outage was triggered by an unexpected pattern of incoming requests that, while not unusually high in volume, each initiated resource-intensive queries. This combination of request frequency and prolonged query execution overwhelmed backend systems, leading to degraded performance and service unavailability. The traffic resembled automated crawling behavior, which our systems were not optimized to handle at that level of query complexity and concurrency. ### **Resolution** To reduce server load and restore service availability, the impacted job site was temporarily disabled. This action immediately alleviated pressure on backend systems and allowed all job boards to recover. The site remained offline over the weekend and was re-enabled on October 6, 2025, after Engineering confirmed that the irregular traffic had subsided. ### **Preventive Actions** To prevent similar incidents in the future, the following measures have been implemented: * **Enhanced Caching:** Improvements to our job site caching mechanisms now limit the ability of similar request patterns to generate excessive server queries. * **Updated Crawling Guidelines:** We have revised our web crawling directives to encourage automated crawlers to reduce the frequency of their requests. This change helps maintain system stability by minimizing the impact of high-volume automated traffic from compliant sources.