Earlier this week AWS Lambda announced they added support to new versions of the Node.js(12), Java(11) and Python(3.8) runtimes.
The most notable differences in these versions on each runtime are:
- assignment expressions syntax ("walrus operator")
- positional-only arguments
- numerous implementation improvements (such as vectorcall).
- and more.
- the var keyword
- a native HTTP client
- TLS 1.3 support
- and more.
- uses V8 7.4
- allows class fields
- added TLS 1.3 support
- and more.
Updating your functions to the new runtimes will allow you to enjoy the improvements and features provided by the new runtimes, but will it come at a performance cost?
Invocation overhead poses a critical performance challenge for FaaS providers and users alike (read more about the it here).
Thanks to FaaSbenchmark and FaaStest.com, our FaaS benchmarking framework and the website that presents it's results (read more about it here) - it's easy to look at invocation overhead patterns, analyze them and draw conclusions.
We will examine the results of the IncreasingLoadLvl3 test, which deploys a function that sleeps for 5 seconds before it returns every time it's invoked. This way, we can see how Lambda handles invocations of a function that already has running instances. We invoke more and more concurrent instances of the same function, so that by the end of the test we have 820 instances of this function up at the same time.
We test each runtime with 5 different functions, each has a different amount of memory allocated to it (128mb, 256mb, 512mb or 1024mb, 1536mb).
Unsurprisingly, Lambda handles our test reasonably well with all runtimes:
All that's left for us to do is to compare the performance of the new runtimes with their older counterparts:
The java11 runtime brings with it a significant performance improvement. The most extreme difference from java8 is the 128mb function:
This is about 40% improvement compared to the last version!
Node.js 12 seems to have more invocation overhead than before. Again, the difference is best seen when comparing the 128mb function:
Here, Node.js 12's invocation overhead is about 21% longer.
Python 3.8's invocation overhead is longer than Python 3.7's by up to a margin of 6%, which is probably not enough to act upon but is an interesting trend to follow nevertheless.
Measuring the mean invocation overhead value gives us the general picture but it's not enough.
We not only want the runtimes' invocation overhead to be low, but we want it to be as consistent as possible.
Faastest features tools to help us check just that. Using the box plot view, we can clearly see that the java11 invocation overheads were very consistent, unlike java8's 1536mb function:
We can also see that we experienced an invocation overhead spike while testing Node.js 12:
When faced with a software update we usually consider new features, stability, security and runtime performance. When it comes to a FaaS runtime we have to consider another factor we usually don't think about - How does the FaaS provider handles the runtime? How does it affect it's performance?
As we showed above, there is no rule of thumb - every runtime is different and every version will be different. But, with faastest.com it's easier than ever to examine those changes and decide what works best for you.
Will you update your Lambda functions? Let us know!