Updates
Resolved
This incident has been resolved.
Monitoring
Zookeeper corruption has been rooted out. Things appear healthier and catching up in all cases.
Identified
We are not to full resolution yet.
Identified
Throughput has improved although behavior of individual partitions remains a problem and is still causing delays in some cases.
Identified
It has been a long day with kafka. We continue to experience instability, causing lag and dropped payloads.
Monitoring
An alternate approach has been applied, we are watching.
Identified
The initial fix was unsuccessful, certain accounts are now substantially delayed.
Monitoring
A fix has been implemented and we are monitoring the results.
Investigating
We are currently investigating this issue.