Resolved incident

[Medium] Issue with Uploads

Box · Major · Started

Duration
41m
Severity
Major
Status
Postmortem

Summary

Major · Postmortem

We recently addressed issues affecting file uploads and other write operations within Box. We would like to take the opportunity to further explain these issues and the steps we have taken to keep them from happening in the future. Between 02:03 PM and 02:24 PM PDT on August 24, 2026, some users may have experienced difficulties while working in Box. During this time, users saw elevated error rates and reduced availability for uploads and other write-oriented actions. The issue occurred when a background user-cleanup process generated a sudden surge of database update activity that was amplified by an unbounded retry loop, which overloaded a database shard and exhausted connection capacity. We restored normal service by applying a rate limit to the cleanup traffic and stopping the offending cleanup workload. After traffic subsided, we removed the temporary limits once systems were stable. In addition, we are making changes to limit background cleanup traffic and improve retry behavior to reduce the chance of recurrence. # Analysis Following this incident, we are now improving how similar types of background jobs are managed and retried, and making adjustments to better prioritize customer-facing operations over these tasks. Specifically, during this issue, rapid retry cycles amplified service friction, highlighting areas where additional safeguards can further protect against similar issues occurring again in the future. We are also reviewing rollout and feature-flag practices to avoid accidental large-scale job generation. # Corrective Actions Box has initiated the following corrective actions: * Implementing bounded exponential-backoff retries for background cleanup jobs. * Enforcing throttling and monitoring on background cleanup traffic so it cannot overwhelm write-path resources. * Introducing priority-aware protections so low-priority background work is shed before it affects customer-facing operations. We are continuously working to improve Box and want to make sure we are delivering the best product and user experience we can. We hope we have provided some clarity here and we would be happy to answer any questions you may still have regarding this matter. Sincerely, The Box Team

View official incident ↗

Event timeline

How this incident unfolded, from the official status page.

  1. Investigating

    Box

    Our team is investigating an issue with Uploads. Users may see errors or slowness when attempting to upload items to Box. We will provide additional information as it becomes available.

  2. Identified

    Box

    Our team has identified the underlying cause of this issue and is working to take remediating steps. We will provide additional updates as they become available.

  3. Identified

    Box

    Our team has taken steps to remediate this issue and is seeing improvement for the Upload service. We are continuing to monitor for any additional impact.

  4. Monitoring

    Box

    Our team is investigating an issue with Uploads. Users may see errors or slowness when attempting to upload items to Box. We will provide additional information as it becomes available.

  5. Monitoring

    Box

    Our team has taken steps to remediate this issue and is seeing improvement for the Upload service. We are continuing to monitor for any additional impact.

  6. Resolved

    Box

    After further monitoring, this incident is now considered resolved. If you continue to experience any issues, please contact Box Support at https://support.box.com.

  7. Postmortem

    Box

    We recently addressed issues affecting file uploads and other write operations within Box. We would like to take the opportunity to further explain these issues and the steps we have taken to keep them from happening in the future. Between 02:03 PM and 02:24 PM PDT on August 24, 2026, some users may have experienced difficulties while working in Box. During this time, users saw elevated error rates and reduced availability for uploads and other write-oriented actions. The issue occurred when a background user-cleanup process generated a sudden surge of database update activity that was amplified by an unbounded retry loop, which overloaded a database shard and exhausted connection capacity. We restored normal service by applying a rate limit to the cleanup traffic and stopping the offending cleanup workload. After traffic subsided, we removed the temporary limits once systems were stable. In addition, we are making changes to limit background cleanup traffic and improve retry behavior to reduce the chance of recurrence. # Analysis Following this incident, we are now improving how similar types of background jobs are managed and retried, and making adjustments to better prioritize customer-facing operations over these tasks. Specifically, during this issue, rapid retry cycles amplified service friction, highlighting areas where additional safeguards can further protect against similar issues occurring again in the future. We are also reviewing rollout and feature-flag practices to avoid accidental large-scale job generation. # Corrective Actions Box has initiated the following corrective actions: * Implementing bounded exponential-backoff retries for background cleanup jobs. * Enforcing throttling and monitoring on background cleanup traffic so it cannot overwhelm write-path resources. * Introducing priority-aware protections so low-priority background work is shed before it affects customer-facing operations. We are continuously working to improve Box and want to make sure we are delivering the best product and user experience we can. We hope we have provided some clarity here and we would be happy to answer any questions you may still have regarding this matter. Sincerely, The Box Team