activity log for US instance is delayed

Event timeline

How this incident unfolded, from the official status page.

1. Identified

   6 months ago

   Honeycomb

   Activity log delay is rising, and is more than an hour behind due to a MySQL replica failover. Estimated recovery by 04:00 PDT.

2. Monitoring

   6 months ago

   Honeycomb

   Backfilling of the past 2h30m of data is now in progress and should complete shortly.

3. Monitoring

   6 months ago

   Honeycomb

   We believe that the activity log should now be caught up.

4. Monitoring

   6 months ago

   Honeycomb

   We are continuing to monitor for any further issues.

5. Monitoring

   6 months ago

   Honeycomb

   All activity log streams are now caught up, except for the query runs table, which should have all data since the start of the incident backfilled by 1600 PDT.

6. Monitoring

   6 months ago

   Honeycomb

   Activity Log recovered at 07:00 UTC-07:00 today (about 6.5h ago).  All streams are caught up.

7. Resolved

   6 months ago

   Honeycomb

   This incident has been resolved.