December 04, 2018
One of the most common pains for users of AWS Lambda is cold starts. Cold starts add unwanted delays to Lambda invocations, and in cases where a Lambda is used inside of a Virtual Private Cloud (VPC), the latency can be as high as several seconds. This practically negates the speed benefits of Lambda functions.
Fortunately, the Lambda team announced at AWS re:Invent 2018 that they are changing the architecture of Lambdas running in a VPC in order to reduce this latency and make Lambdas start much faster.
A VPC creates a network perimeter around AWS resources, restricting their access to outside resources such as the internet and other AWS services. In order to run a Lambda inside of a VPC, an Elastic Network Interface (ENI) must be created, an IP address must be allocated to the ENI, then the ENI is attached to the Lambda. According to the AWS documentation, this incurs an additional startup penalty that can take as long as 10 seconds.
Each concurrent invocation of a Lambda function also requires a separate ENI, leading to scaling issues. Not only does each VPC subnet have a limited number of IP addresses, but the number of concurrent Lambda invocations can exceed your ENI limit. The AWS documentation states that “you must make sure that your VPC has sufficient ENI capacity to support the scale requirements of your Lambda function. You can use the following formula to approximately determine the ENI requirements: Projected peak concurrent executions * (Memory in GB / 3GB).”
This combination of lengthy cold start times and limited scalability has led developers and software architects to choose different solutions over Lambda, especially for user-facing or time-sensitive applications. But with the recent announcements from the Lambda team, these problems may soon be resolved.
During the "AWS Lambda Under the Hood" session – which was, in my opinion, the best session at re:Invent 2018 – @MarcBrooker and Holly Mesrobian from the Lambda engineering team shared that a new architecture is coming to Lambdas VPCs. Each VPC will include a dedicated ENI connected to a remote Network Address Translation (NAT). Functions will share the same VPC connection and connect to the NAT server using a secure tunnel. This lets Lambdas quickly and securely connect to the VPC and access restricted subnets using an existing connection. This also means that functions inside the VPC will share the same IP address.
This new architecture has two benefits:
We'll have to wait for the rollout of the new architecture to see how much this improves performance, but from what we've seen so far, it will have a significant impact on cold start times.
This new architecture doesn't just promise to improve performance, but it also makes configuring Lambdas in a VPC much more user-friendly. This has the potential to make Lamba more appealing, especially for enterprises that run their applications in VPCs. We're excited to see this new change and can't wait to see how well it reduces cold start times.