Reducing Clojure Lambda Cold Starts Part 10

Rust seems to be at the height of the hype cycle right now even among functional programming enthusiasts. Although it's not a true functional programming language, due to not having first-class support for immutable data structures, its ownership model does provide the some of same safety guarantees as immutability. Its core library also comes stock with a lot of the high-level programming features I can't live without, although supposedly still being really fast with its "no cost abstractions". It all seems too good to be true. Let's investigate, with a particular eye towards Lambda.

I create a new project with:

>> cargo new tax_engine_experiments_rust --bin

Modify src/main.rs:

use lambda_runtime::{handler_fn, Context, Error};
use serde_json::{json, Value};

#[tokio::main]
async fn main() -> Result<(), Error> {
    let func = handler_fn(func);
    lambda_runtime::run(func).await?;
    Ok(())
}

async fn func(event: Value, _: Context) -> Result<Value, Error> {
    println!("EVENT: {}", event);
    Ok(json!({ "message": format!("Hello Event, {}!", event) }))
}

And modify Cargo.toml to look like this:

[package]
name = "tax_engine_experiments_rust"
version = "0.1.0"
edition = "2021"
autobins = false

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
lambda_runtime = "0.4.1"
tokio = { version = "1.0", features = ["macros", "io-util", "sync", "rt-multi-thread"] }
serde = "^1"
serde_json = "^1"
serde_derive = "^1"

[[bin]]
name = "bootstrap"
path = "src/main.rs"

Then follow the instructions here to compile using Docker, but slightly differently:

>> LAMBDA_ARCH="linux/arm64"
>> RUST_TARGET="aarch64-unknown-linux-gnu" 
>> RUST_VERSION="latest" 
>>docker run \
  --platform ${LAMBDA_ARCH} \
  --rm --user "$(id -u)":"$(id -g)" \
  -v "${PWD}":/usr/src/myapp -w /usr/src/myapp rust:${RUST_VERSION} \
  cargo build --release --target ${RUST_TARGET}

Then I create lambda.zip:

cp ./target/aarch64-unknown-linux-gnu/release/bootstrap ./bootstrap && zip lambda.zip bootstrap && rm bootstrap

Add a test event file:

{
  "Records": [
    {
      "messageId": "19dd0b57-b21e-4ac1-bd88-01bbb068cb78",
      "receiptHandle": "MessageReceiptHandle",
      "body": "{\"bucket\": \"tax-engine-experiments-2-transactionsbucket-78gg1f219mel\", \"key\": \"test.json\"}",
      "attributes": {
        "ApproximateReceiveCount": "1",
        "SentTimestamp": "1523232000000",
        "SenderId": "123456789012",
        "ApproximateFirstReceiveTimestamp": "1523232000001"
      },
      "messageAttributes": {},
      "md5OfBody": "{{{md5_of_body}}}",
      "eventSource": "aws:sqs",
      "eventSourceARN": "arn:aws:sqs:us-east-1:123456789012:MyQueue",
      "awsRegion": "us-east-1"
    }
  ]
}

Add a template.yml:

AWSTemplateFormatVersion: "2010-09-09"

Transform:
- "AWS::Serverless-2016-10-31"

Parameters:

  TransactionsBucket:
    Type: String
    Default: tax-engine-experiments-2-transactionsbucket-78gg1f219mel

  CalculationsBucket:
    Type: String
    Default: tax-engine-experiments-2-calculationsbucket-aivptjt1j82w

Resources:

  RunRustCalculationsQueue:
    Type: AWS::SQS::Queue
    Properties:
      QueueName: !Sub "${AWS::StackName}-run-calcs-queue-rust"
      VisibilityTimeout: 5400

  RunCalculationsRust:
    Type: AWS::Serverless::Function
    Properties:
      Architectures:
        - arm64
      FunctionName: !Sub "${AWS::StackName}-run-calcs-rust"
      Handler: none
      Runtime: provided.al2
      CodeUri: lambda.zip
      Timeout: 900
      MemorySize: 512
      Policies:
        - AWSLambdaBasicExecutionRole
        - S3ReadPolicy:
            BucketName: !Ref TransactionsBucket
        - S3WritePolicy:
            BucketName: !Ref CalculationsBucket
      Environment:
        Variables:
          RUST_BACKTRACE: 1
          TRANSACTIONS_BUCKET: !Ref TransactionsBucket
          CALCULATIONS_BUCKET: !Ref CalculationsBucket
      Events:
        SQSEvent:
          Type: SQS
          Properties:
            Queue: !GetAtt RunRustCalculationsQueue.Arn
            BatchSize: 1

Note: I tried using serverless-rust as it seemed like a quick way to get up and running with Rust Lambdas, but I use AWS SSO and it doesn't work with Serverless without a bunch of uncomfortable hacks, so I reverted to SAM.

Then invoke with SAM Local:

>> sam local invoke -e test-event.json

Looks good:

Invoking none (provided.al2)
Decompressing /Users/larry/Documents/code/tax_engine_experiments_rust/lambda.zip
Skip pulling image and use local one: public.ecr.aws/sam/emulation-provided.al2:rapid-1.35.0-arm64.

Mounting /private/var/folders/2w/hrc_hrn52nq8n80c7j64tb0c0000gp/T/tmpru54maap as /var/task:ro,delegated inside runtime container
START RequestId: 6706ea25-0135-424f-af83-b8ac0be0eaac Version: $LATEST
EVENT: {"Records":[{"attributes":{"ApproximateFirstReceiveTimestamp":"1523232000001","ApproximateReceiveCount":"1","SenderId":"123456789012","SentTimestamp":"1523232000000"},"awsRegion":"us-east-1","body":"{\"bucket\": \"tax-engine-experiments-2-transactionsbucket-78gg1f219mel\", \"key\": \"test.json\"}","eventSource":"aws:sqs","eventSourceARN":"arn:aws:sqs:us-east-1:123456789012:MyQueue","md5OfBody":"{{{md5_of_body}}}","messageAttributes":{},"messageId":"19dd0b57-b21e-4ac1-bd88-01bbb068cb78","receiptHandle":"MessageReceiptHandle"}]}
END RequestId: 6706ea25-0135-424f-af83-b8ac0be0eaac
REPORT RequestId: 6706ea25-0135-424f-af83-b8ac0be0eaac    Init Duration: 1.46 ms    Duration: 86.13 ms    Billed Duration: 100 ms    Memory Size: 512 MB    Max Memory Used: 512 MB    
{"message":"Hello Event: {\"Records\":[{\"attributes\":{\"ApproximateFirstReceiveTimestamp\":\"1523232000001\",\"ApproximateReceiveCount\":\"1\",\"SenderId\":\"123456789012\",\"SentTimestamp\":\"1523232000000\"},\"awsRegion\":\"us-east-1\",\"body\":\"{\\\"bucket\\\": \\\"tax-engine-experiments-2-transactionsbucket-78gg1f219mel\\\", \\\"key\\\": \\\"test.json\\\"}\",\"eventSource\":\"aws:sqs\",\"eventSourceARN\":\"arn:aws:sqs:us-east-1:123456789012:MyQueue\",\"md5OfBody\":\"{{{md5_of_body}}}\",\"messageAttributes\":{},\"messageId\":\"19dd0b57-b21e-4ac1-bd88-01bbb068cb78\",\"receiptHandle\":\"MessageReceiptHandle\"}]}!"}%

Deploying and running in the Lambda console I get:

Screen Shot 2021-12-28 at 6.02.19 PM.png

It does seem to be very fast:

Duration: 0.88 ms    Billed Duration: 17 ms    Memory Size: 512 MB    Max Memory Used: 13 MB    Init Duration: 15.95 ms

We'll see how it does with an S3 dependency and our calculations workload in my next post.

Reducing Clojure Lambda Cold Starts Part 10 - Rust?

Comments

Reducing Clojure Lambda Cold Starts

Reducing Clojure Lambda Cold Starts Part 11 - Rust Part 2

More from this blog

LocalStack Testing in Clojure

SQS To DynamoDB Tuning - Writing Data to DDB in Rust and Typescript

SQS To DynamoDB Tuning - Setting Up Rust And Typescript Lambdas

Reducing Clojure Lambda Cold Starts Part 11 - Rust Part 2

Command Palette

Comments

Reducing Clojure Lambda Cold Starts

Reducing Clojure Lambda Cold Starts Part 11 - Rust Part 2

More from this blog