Activity log outage in production US

Resolved

Minor impact

Started

Lasted 25h 23m

View official incident ↗

Updates

  1. Resolved

    As of 23:00 UTC, all systems are back to synchronized.

  2. Monitoring

    We are continuing to work on catching up late data from yesterday night's replication issues and expect to complete this process in approximately 6 hours.

  3. Monitoring

    Due to issues with replication streams, the activity log will have a gap in its data. We estimate that a little over 6 hours of content will be missing, from 20:27 UTC until 2:43 UTC. We do not expect missing data to be backfilled. No other Honeycomb data or feature has been impacted. We are currently working on catching up on late data after this window.

  4. Identified

    We are continuing to work on a fix for this issue.

  5. Identified

    We have identified the cause and are working to resolve the issue.