Smaller runtime and optimized internals

Mitu899

Deployed to a model with a relatively large architecture before we started working on improvements. The next graph shows the significant improvement in startup performance we could see after we made performance improvements. The improvement. Now we will delve deeper and discuss the various changes that have brought us to such a state of great improvement. A similar protocol was previously used to communicate between the client and the query engine, based on the more efficient Wire protocol. This introduces some quirks that affect performance especially during cold starts in a serverless environment. During our performance exploration we noticed that the current implementation adds considerable memory overhead especially for larger patterns.

One of our solutions to mitigate this problem is to completely redesign our wired protocol. Using it we can significantly improve the efficiency of communication between the client and the qu photo editing servies ery engine. We released this feature in the build with the preview feature flag. Before we start any performance improvement work, the average cold start request is shown below. Before enabling the preview feature, the chart is shown below. After a lot of feedback from users and extensive testing, we are happy to announce that the protocol is now generally available and client-side. The default wire protocol that will be used behind the scenes. If you're interested in more details, we've written an extensive blog post that goes into depth about the changes we've made to improve client startup performance.



How we use Serverless Cold Start to make serverless cold starts up to 2x faster.  In addition to changing the protocol we also made a number of changes that affected performance. With the new 1000x1000 based wire protocol becoming the default protocol we took the opportunity to clean up the dependencies. This includes cutting the dependencies in half and removing previous similar protocol implementations. This reduces the execution time and amount of memory used by the client. We also optimized the internal structure of the query engine. Specifically the part responsible for converting and establishing a database connection when the query engine starts. Additionally we now defer generating strings for many type names in query mode which improves memory usage and significantly improves runtime performance.