Increased cold start latency in us-west-2

Resolved

Started

Lasted 15h 29m

View official incident ↗

Updates

  1. Resolved

    This incident has been resolved. We moved away most of the load in this region and compute start latency is back to normal.

  2. Investigating

    Identified: (06:00) Seeing increased pod start latency due to the kube-proxy sync taking more time. Besides elevated start time we do not expect any operational problems. Identified: (20:19) • Cold start latency in us-west-2 is about 19 seconds, which is a lot more than the target (5 s). • We have considered several different fixes and mitigations, but all of them involve some new development and risk. On balance, we determined that it is best to not rush with a fix, and live with the increased cold start latency until we have tested the fix thoroughly. • ETA for a fix is 1-2 days.