Session Replay event ingestion: processing delays

Resolved

Started

Lasted 1h 27m

View official incident ↗

Updates

  1. Resolved

    An offline batch job overloaded a cache that session recording ingestion pipeline relied on for processing. This resulted in some session recording ingestion delays. Moving the batch job off of hot path caches and ensuring the ingestion pipeline is more resilient to cache failures will keep this from happening again.

  2. Monitoring

    Session replay ingestion lag has fully recovered

  3. Identified

    The system is starting to see recovery after scaling up a cache that is used in the session replay ingestion pipeline.

  4. Investigating

    We’ve identified processing delays in the event ingestion pipeline. Events may take longer than usual to appear in the product. Data is not lost but may not show in PostHog apps and queries until the processing delay is resolved.