Updates
Resolved
We can confirm that the fix was applied uniformly to all impacted and potentially impacted environments; no teams should see ingest errors related to this issue anymore.
Monitoring
We are continuing to work on the systemic solution and monitor. We'll leave the incident open until we're able to confirm the issue won't recur.
Monitoring
The manual fix has been applied to all environments.
Monitoring
We've applied the manual fix to nearly all affected datasets (less than 1% remaining). The systemic solution is still in progress.
Identified
We've identified a manual fix and are deploying that by hand to each affected environment, while also investigating a systemic solution.
Identified
We have further narrowed down that only teams on Classic environments are currently seeing issues. We are figuring out mitigation mechanisms at the moment.
Identified
We have confirmed reports of some teams getting issues ingesting their data. We have identified a probable source behind this behavior and are currently trying to correct it.