Live outage feed

Outages and recoveries from the services you depend on.

Earlier incidents

40 incidents

Back to latest

Aug 11, 2026

  1. HubSpotSome HubSpot tools may be unavailable for some users

    Between 03:16 PM EDT (UTC -04:00) and 03:25 PM EDT (UTC -04:00), many customers in all affected regions experienced issues accessing the CRM Record page. This was caused by a server impairment from a configuration update. As of 03:25 PM EDT (UTC -04:00), our CRM Record page are working properly and the incident has been fully resolved. HubSpot conducts a thorough review after each incident to understand the cause and prevent it from happening again. Learn more about HubSpot's commitment to reliability at www.HubSpot.com/reliability. The information on this page reflects our understanding of the incident and impact at the time of the update.

    Resolved

  2. CircleCIDelayed pipeline updates and pipeline processing failures

    Between 13:50 UTC and approximately 17:30 UTC on August 11, 2026, some customers experienced delays of up to an hour in pipeline status updates and in notification delivery. Between 16:40 UTC and 17:00 UTC within that same window, a small number of customers also saw newly created pipelines fail to process, appearing in an errored state in the UI and API. The issue has been resolved and all affected functionality has returned to normal. Customers whose pipelines errored during that window can retrigger them. We thank you for your patience while our team worked on implementing a fix.

    Resolved

  3. GitHubIncident with GraphQL API Requests

    On July 26, 2026 at 21:34 UTC we began seeing intermittent errors on the GitHub GraphQL API. A subset of GraphQL API requests returned HTTP 502 errors in short bursts. During the impact window an average of 0.09% of GraphQL API requests in the affected region failed, with a peak of 0.50% of requests failing during the worst two-minute period at 03:02 UTC on July 27. Requests that failed generally succeeded when retried, and no data was lost or altered. Other GitHub services were not affected.<br /><br />The errors were traced to a single group of servers handling a share of GraphQL API traffic. Application processes on that group intermittently closed connections before completing responses. Impact ended at 03:52 UTC on July 27 when those processes were replaced, and we resolved the incident at 04:09 UTC on July 27 after confirming error rates had returned to normal.<br /><br />We are still investigating why those processes closed connections, and that work is being carried out by the team that owns the underlying compute platform. In the meantime we are adding detection and automated mitigation for when a single group of servers behaves differently from its peers.

    Resolved

  4. UpCloudCloud Server Deployment Error

    Our team has confirmed the full resolution of the issue. All affected services should now be functioning normally. If you continue to experience any further issues, please contact our Support team via Live Chat or email support@upcloud.com Thank you for your patience, and we apologize for any inconvenience caused.

    Resolved

  5. GitHubDegraded availability GPT 5.6 Luna

    On August 1st, 2026, the GPT-5.6 Luna model in GitHub Copilot experienced degraded availability in intermittent time intervals between ~08:05 UTC and ~16:30 UTC. Specifically the timeframes observed were 10:00-10:20 UTC, 10:45-11:50 UTC, 13:00-14:25 UTC, and 16:00-16:30 UTC. During this time, requests to GPT-5.6 Luna in Copilot chat and IDE surfaces frequently failed or timed out. This was caused by an issue with an upstream model provider. Other Copilot models were not affected, and users could continue working by selecting another model or 'Auto'. Availability for GPT-5.6 Luna fully recovered once the provider resolved their outage at 16:30 UTC.

    Resolved

  6. Box[Medium] Issues with Box Hubs

    We recently addressed an issue affecting Box Hubs. We’d like to take this opportunity to explain what happened and the steps we’ve taken to prevent it from happening again. Between 06:50 AM PDT to 07:26 AM PDT on July 23, 2026, some users may have experienced difficulties while working in Box. During this time, users attempting to access Box Hubs were unable to load the Hubs gallery and instead encountered a "Not Found" page. The issue occurred as a result of a recent frontend dependency update, which introduced a package version conflict that prevented the Hubs module from loading correctly. We were able to resolve the issue by rolling back the frontend application to the previous stable version, which fully restored access to Box Hubs. In addition, we are working to improve our automated pre-release testing processes to ensure that dependency updates are validated across all affected modules before reaching production, and we are enhancing our monitoring and alerting capabilities to detect and surface similar issues more quickly, to prevent similar issues from occurring in the future. # **Analysis** At the technical level, a shared package used by the core web application and the Hubs page module fell out of version alignment when the core application upgraded the dependency without a corresponding update in the Hubs module. This caused the Hubs page to crash at load time. The regression was not caught before reaching production because pre-release automated tests were scoped to core application features and did not cover dependent page modules. Once in production, detection was slowed by monitoring gaps, meaning the incident was ultimately surfaced by synthetic monitoring rather than production observability tooling. # **Corrective Actions** Box has initiated the following corrective actions: * Improving frontend error monitoring to clearly attribute client-side errors to specific pages, enabling faster detection of page-level failures in production. * Enhancing module-level observability and alerting to detect load-time failures for modules. This will ensure we are fully aware of any issues in the integration of sub-modules \(such as Hubs\) with the core web app as early as possible. * Expanding pre-release automated smoke tests to cover all page modules dependent on the core web application, ensuring that package version conflicts are caught in the release pipeline before reaching production. * Improving staging environment alerting to surface potential regressions earlier in the release process, before changes are promoted to production. We are continuously working to improve Box and want to make sure we are delivering the best product and user experience we can. We hope we have provided some clarity here and we would be happy to answer any questions you may still have regarding this matter.  Sincerely, The Box Team

    Resolved

  7. GitHubIncident with Actions

    On August 6, 2026, between 15:05 UTC and 00:14 UTC on August 7, GitHub Actions experienced degraded availability. During the incident, workflow runs failed or remained queued for an extended period of time. Customers using both GitHub-hosted and self-hosted runners were affected. At peak, 71% of workflow runs experienced infrastructure failures and 75% of the remaining workflow runs were delayed by more than 5 minutes. <br /><br />The incident was triggered by a routine deployment to an internal Actions service responsible for processing events and generating Actions jobs. The deployment exposed an existing capacity and concurrency weakness. As pods were replaced during the deployment, remaining capacity became saturated, causing services to crash and triggering a cascading impact across multiple clusters and downstream services. <br /><br />These services recovered at 17:00 after expanding capacity, throttling incoming webhook-triggered work to allow the system to recover, and increasing processing capacity for the backlog of affected events. <br /><br />As the incident progressed, a backlog of work accumulated across the systems responsible for assigning jobs to runners. Due to a latent bug in one of the services responsible for job assignment, runners were getting assigned jobs that were no longer valid and then getting stuck retrying those jobs, preventing them from picking up valid work. <br /><br />This second stage of impact was mitigated by deploying changes to prevent runners from repeatedly attempting to acquire invalid jobs. These mitigations allowed the accumulated queues to drain and Actions to recover to normal operation. <br /><br />Some Actions Runner Controller (ARC) runners remained stuck after the incident. A mitigation deployed during the incident inadvertently affected these runners, causing some to remain offline until they were manually recovered. We subsequently rolled back the change and are adding automatic recovery in upcoming Runner and ARC releases. <br /><br />Some jobs created during the incident were also left stuck unable to be retried or canceled. CLI and UI solutions for customers to address these were shared at https://github.com/orgs/community/discussions/204152#discussioncomment-17946043. <br /><br />To prevent recurrence, we are making improvements to deployment and capacity safeguards for the affected services, strengthening monitoring for the conditions that preceded the incident, improving the resiliency and recovery of queued work and runner assignment, and adding automatic recovery for self-hosted runners affected by similar failure conditions. We are also making additional improvements to reduce the risk of cascading failures and accelerate recovery during large-scale Actions disruptions.

    Resolved

  8. SnowflakeINC20000128

    Snowflake Engineering has completed the postmortem of this service incident. A detailed Root Cause Analysis \(RCA\) is available on the Snowflake Community site: [https://community.snowflake.com/s/article/INC20000128](https://community.snowflake.com/s/article/INC20000128)

    Resolved

  9. SnowflakeINC20000129

    Snowflake Engineering has completed the postmortem of this service incident. A detailed Root Cause Analysis \(RCA\) is available on the Snowflake Community site: [https://community.snowflake.com/s/article/INC20000129](https://community.snowflake.com/s/article/INC20000129).

    Resolved

Aug 10, 2026

  1. GitHubIncident with Pages - Deployment Lag

    On August 6, 2026, at 07:00 UTC, a configuration change inadvertently reduced the capacity of the service that processes GitHub Pages deployments. As traffic increased over the following hours, latency in the deployment pipeline progressively increased. <br /><br />At 12:09 UTC, latency crossed the alerting threshold and the team began investigating. We reverted the invalid configuration and applied additional mitigations, including reducing status deployment processing to lower the load on our Redis cluster. Latency returned to normal levels at 15:40 UTC. <br /><br />Customer impact occurred from 11:34 to 15:32 UTC. During this period, we failed to process approximately 128,000 deployments. <br /><br />We have updated our alerts to detect elevated processing latency sooner and to notify us immediately when latency causes deployment processing failures. We've confirmed this incident was not fully captured by our availability metrics. In the coming days, we'll update how GitHub Pages availability is measured so incidents like this are accurately reflected going forward.

    Resolved

  2. HarnessAIDI Dashboards – Degraded Performance

    ## Summary Customers on Prod1, Prod2, and Prod3 clusters experienced intermittent widget load failures and increased load times when accessing AIDI 2.0 dashboards on July 22, 2026. Not all widgets were affected simultaneously the issue manifested as sporadic failures rather than a full outage. No customer data was lost. SEI 1.0 customers were not impacted. ## Root Cause Over time, a routine database maintenance process failed to run on certain tables in our analytics database, causing those tables to accumulate a large volume of internal metadata used to track deleted records. When the database planned queries against these tables, it loaded all of this accumulated metadata into memory, causing memory usage on the affected nodes to spike repeatedly. These repeated spikes triggered an automatic safety mechanism that restarts a node when it detects excessive memory pressure, and the affected nodes began restarting in a loop as a result. This caused intermittent, degraded query performance on AIDI 2.0 dashboards for the duration of the incident. ## Impact Customers on Prod1, Prod2, and Prod3 clusters may have experienced intermittent widget load failures or increased load times on AIDI 2.0 dashboards. **Duration:** July 22, 2026, 07:58 PDT – 16:16 PDT \(~8 hours 18 minutes\), with intermittent widget failures; system was restarted and under active monitoring from 08:25 PDT onward. ### What was not impacted? * Data ingestion and processing * SEI 1.0 customers * Integrations and metadata flows No customer data was lost. ## Remediation Upon identifying the issue, the affected database nodes were restarted at 08:25 PDT, which restored initial stability. We continued to monitor the system closely, and when intermittent degradation was still observed afterward, we applied several additional fixes: * Adjusted database configuration settings to limit the amount of memory used for processing accumulated metadata, and tuned query-planning settings to reduce memory pressure. * Ran cleanup jobs to reduce the backlog of accumulated metadata on the affected tables. * Increased capacity on the affected database nodes to provide additional headroom. These changes progressively stabilized the system, and the incident was fully resolved at 16:16 PDT. ## Action Items To prevent recurrence, we are implementing the following: 1. We have upgraded the backend which includes underlying improvements that handle memory spikes caused by excessive delete files. 2. We have rolled out automated compaction jobs for newly introduced tables to prevent delete file accumulation going forward.

    Resolved

  3. SpeechmaticsBatch SaaS AU - Partial Degradation

    We identified an issue that caused degraded performance in our Batch SaaS AU region between 14:00 - 16:00 UTC. Customers may see delays processing some of the jobs. Our team has restored service to normal performance levels. We apologise for any inconvenience.

    Resolved

Older incidents