Reducing Clojure Lambda Cold Starts Part 4

ClojureScript Lambdas on Node seem promising so far, with an average cold start time of 182.5228 ms vs. 2.6567039 seconds for a similarly bare-bones Clojure Lambda on the JVM. But how will they compare when it comes to performing more realistic workloads?

Add AWS S3 SDK

I'll configure the SDKs and then use listBuckets to test the configuration.

ClojureScript

In src/cljs/tax/core.cljs:

(ns tax.core
  (:require [cljs.core.async :as async :refer [<!]]
            [cljs.core.async.interop :refer-macros [<p!]]
            ["aws-sdk" :as aws])
  (:require-macros [cljs.core.async.macros :refer [go]]))

(def client (aws/S3.))

(defn list-buckets []
  (.promise (.listBuckets client)))

(defn handler [event context callback]
  (go (let [result (<p! (list-buckets))]
          (callback nil result))))

Clojure

In deps.edn:

{:paths ["src/clj"]
 :deps {software.amazon.awssdk/s3 {:mvn/version "2.17.100"}}
 :aliases {:build {:deps {io.github.clojure/tools.build {:tag "v0.7.2" :sha "0361dde"}}
                   :ns-default build}
           :profile {:extra-paths ["dev/clj"]
                     :deps {software.amazon.awssdk/sqs {:mvn/version "2.17.100"}
                            software.amazon.awssdk/sso {:mvn/version "2.17.100"}}}}}

In src/clj/tax/core.clj

(ns tax.core
  (:import (software.amazon.awssdk.services.s3 S3Client)
           (software.amazon.awssdk.services.s3.model ListBucketsRequest))
  (:gen-class
   :methods [^:static [calculationsHandler [Object] Object]]))

(defn list-buckets []
  (let [s3 (-> (S3Client/builder) (.build))
        req (-> (ListBucketsRequest/builder) (.build))]
    (.listBuckets s3 req)))

(defn -calculationsHandler [event]
  (str (list-buckets)))

And in template.yml, add some buckets and allow the Lambdas to access them:

AWSTemplateFormatVersion: "2010-09-09"

Transform:
- "AWS::Serverless-2016-10-31"

Resources:

  RunClojureScriptCalculationsQueue:
    Type: AWS::SQS::Queue
    Properties:
      QueueName: !Sub "${AWS::StackName}-run-calcs-queue-cljs"
      VisibilityTimeout: 5400  

  RunClojureCalculationsQueue:
    Type: AWS::SQS::Queue
    Properties:
      QueueName: !Sub "${AWS::StackName}-run-calcs-queue-clj"
      VisibilityTimeout: 5400

  TransactionsBucket:
    Type: AWS::S3::Bucket

  CalculationsBucket:
    Type: AWS::S3::Bucket

  RunCalculationsCLJ:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: !Sub "${AWS::StackName}-run-calcs-clj"
      Handler: tax.core::::calculationsHandler
      Runtime: java11
      CodeUri: target/tax-engine-0.1.0-standalone.jar
      Timeout: 900
      MemorySize: 512
      Policies:
        - AWSLambdaBasicExecutionRole
        - S3ReadPolicy:
            BucketName: !Ref TransactionsBucket
        - S3WritePolicy:
            BucketName: !Ref CalculationsBucket
      Environment:
        Variables:
          TRANSACTIONS_BUCKET: !Ref TransactionsBucket
          CALCULATIONS_BUCKET: !Ref CalculationsBucket
      Events:
        SQSEvent:
          Type: SQS
          Properties:
            Queue: !GetAtt RunClojureCalculationsQueue.Arn
            BatchSize: 1                

  RunCalculationsCLJS:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: !Sub "${AWS::StackName}-run-calcs-cljs"
      Handler: index.handler
      Runtime: nodejs14.x
      CodeUri: target/lambda/calcs
      Timeout: 900
      MemorySize: 128
      Policies:
        - AWSLambdaBasicExecutionRole
        - S3ReadPolicy:
            BucketName: !Ref TransactionsBucket
        - S3WritePolicy:
            BucketName: !Ref CalculationsBucket
      Environment:
        Variables:
          TRANSACTIONS_BUCKET: !Ref TransactionsBucket
          CALCULATIONS_BUCKET: !Ref CalculationsBucket
      Events:
        SQSEvent:
          Type: SQS
          Properties:
            Queue: !GetAtt RunClojureScriptCalculationsQueue.Arn
            BatchSize: 1

Deploying, and running tests in the console, I get:

Clojure:

Init duration: 2864.75 ms Duration: 11055.58 ms

ClojureScript:

Init Duration: 494.70 ms Duration: 1114.63 ms

I was not expecting that! The init duration for CLJS went up significantly while CLJ stayed the same, but the difference in execution time is surprising! I'll run my SQS blaster on each of the two to get more rigorous results.

Screen Shot 2021-12-24 at 10.15.10 PM.png

Screen Shot 2021-12-24 at 10.16.01 PM.png

Summarizing the results:

Clojure:

Average Init Duration: 3037.3744 Average Duration: 712.381

ClojureScript:

Average Init Duration: 454.599 Average Duration: 144.5778

These results are very interesting in several ways. The difference between that first invocation of the CLJ version and the average is quite stark. I need to investigate what is going on there. Also, I would have expected the CLJ version to be much faster after load than the CLJS version, but it is nearly 5 times slower. Another surprising thing is that just adding the dependency on 'aws-sdk' and statically initializing the S3 client increased the init duration by more than double.

This leaves me with several things to investigate:

Was that first cold start with the CLJ Lambda just a statistical anomaly, or am I missing something?
Can I reduce the cold start on the CLJS version by perhaps using the AWS SDK V3 that is supposed to be more modularized?
What are the performance characteristics of the equivalent Java and JavaScript Lambdas? Maybe Clojure and ClojureScript are way worse than their native counterparts.
What are the performance characteristics for different memory sizes? Maybe the CLJ version is struggling with 512 memory and could benefit from a larger memory size.

I'll investigate 3 in my next post.

Reducing Clojure Lambda Cold Starts Part 4 - JVM vs Node Performance

Comments