What is a cold start?
In AWS Lambda, instances are added and removed dynamically depending on how many requests each function receives. When a new instance handles its first request, it incurs a delay known as a cold start.
Notice how I said "a new instance handles its first request" and not "a function handles its first request". Once you receive enough simultaneous requests, an individual AWS Lambda instance isn't enough, and another one spins up, also incurring a cold start delay.
No one really knows exactly how much idle time it takes for an AWS Lambda instance to incur a cold start. You tend to see cold starts at 5 minutes of idle time, but on average it seems to take around 10 minutes.
The larger your AWS Lambda package size, the longer the cold starts will be.
How long do cold starts take?
A good example is our uptime monitoring service, OnlineOrNot. The web server running our app is hosted on AWS Lambda. Our cold starts look like this:
The small blue dots in the chart above show how long a message takes to go one way to a cold AWS Lambda instance. With the size of our AWS Lambda functions, a cold start in AWS Lambda takes roughly 400ms.
How do I avoid cold starts?
<ShamelessPlug>Using an uptime monitor like OnlineOrNot can actually help you here. Below is a chart of our web server's latency while OnlineOrNot monitors itself.
As OnlineOrNot's uptime monitor visits our web server at least every 5 minutes, we never incur a cold start. While there are still spikes when a particularly meaty request comes through (we use server-side rendering), on average our latency is only 34ms.
It isn't possible to completely avoid cold starts though. If we had a massive influx of traffic (fingers crossed!), we would begin to see a large number of cold starts as our function scales up the number of instances it needs.
The best we can do is minimise the symptoms. Things like keeping at least one instance warm, and reducing AWS Lambda function size (through fewer dependencies for example) do wonders for your response time.