AWS Lambda Function from Docker Image

Varokas Panusuwan

Jan 11, 2021 • 5 min read

Since Dec 1, 2020, AWS Lambda allow developers to uses any docker images to be executed as lambda functions. This obviously bring tons of benefits and conveniences.

To me, three most compelling reasons are:

Simplified Dependencies Management. No longer needed to pack dependencies in zip file. This was especially annoying for Java and Python.

Better Local Testing . Since this is docker based. The code we run on our machine is exactly the same running in AWS. Say – If a role and permission is misconfigured, the function blows up locally rather than later in prod.

Greater Size Limit. Instead of 50MB zip file limit, Lambda allows for up to 10GB container image size. This is great news for Data Scientist around the world deploying their 500MB+ models.

Steps

Create a docker image that can be run on lambda
Upload to Amazon ECR
Create Function

The follow steps shows detail for Python.

Find a base image

We could use any correctly created custom images. It is far easier to based off Amazon provided images. We will end up with images that behaves very close to how AWS runs in production today when deploying zip file. (CloudWatch Logging and metrics exported and all).

Deploy Python Lambda functions with container images - AWS Lambda

Deploy your Python Lambda function code as a container image using an AWS provided base image or the runtime interface client.

AWS Lambda

Create a custom image

First, Create the app code (app.py). This should looks exactly like a normal lambda function. We'd need a handler function which accepts an event and lambda context, returning function result.

def handler(event, context):
    return {"hello":"world"}

app.py

Copy the function into our docker image. Run the function with <file_name>.<function_name> in CMD.

FROM public.ecr.aws/lambda/python:3.8

COPY app.py ./
CMD ["app.handler"]

Dockerfile

Build our image

$ docker build . -t lambda-docker-python

Running and Testing

These steps should be very familiar to anybody who has been using docker.

docker run -p 9000:8080 lambda-docker-python:latest

Now we could invoke our functions using curl. Per Lambda Runtime Interface Client library doc, the correct route to the deployed handler is /2015-03-31/functions/function/invocations . (Not sure why they impose this convention though – but you'll get 404 otherwise).

This means we could invoke our function by this curl

$ curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'
{"hello": "world"}

### If we look at docker run logs, we'd see a very familiar lambda logging -- this is great!
..
START RequestId: 826b34b9-498e-4468-9e4c-ffa7db08b050 Version: $LATEST
END RequestId: 826b34b9-498e-4468-9e4c-ffa7db08b050
REPORT RequestId: 826b34b9-498e-4468-9e4c-ffa7db08b050	Init Duration: 0.43 ms	Duration: 165.74 ms	Billed Duration: 200 ms	...
..

Adding Dependencies

In my opinion, this is the most compelling reason to use Docker image based lambda. We do not need to do dependencies packing gymnastics to make it work. We just define and download it like we would in local machine/our docker.

For python, that means creates a requirements.txt and invoke a pip install on the file.

requests==2.25.1

requirements.txt

FROM public.ecr.aws/lambda/python:3.8

COPY requirements.txt ./
RUN pip install -r requirements.txt

COPY app.py ./
CMD ["app.handler"]

modified Dockerfile

import requests

def handler(event, context):
  r = requests.get("https://api.covidtracking.com/v1/us/daily.json")
  j = r.json()

  return [ {"states": r["states"], "positive": r["positive"]} for r in j]

modified app.py

Create ECR Repository

Follow the manual steps or uses CDK. Optionally we may want to keep only last 10 images instead of indefinitely keeps all versions.

ecr_repo2 = ecr.Repository(self, "lambda-docker-python", repository_name="lambda-docker-python")
ecr_repo2.add_lifecycle_rule(max_image_count=10)

Note the created repository id, which would be in form of <account>.dkr.ecr.<region>.amazonaws.com/<repo_name>. We need it to login and upload

Upload Image to ECR

First, Logging into ECR requires AWS CLI to obtain password. We can then pipe into docker login. Per instruction here. Replace 820792572713 with your aws account number.

$ aws ecr get-login-password | docker login --username AWS --password-stdin 820792572713.dkr.ecr.us-west-2.amazonaws.com
Login Succeeded

Then we can tag and upload the image.

$ docker tag lambda-docker-python:latest 820792572713.dkr.ecr.us-west-2.amazonaws.com/lambda-docker-python:latest

$ docker push 820792572713.dkr.ecr.us-west-2.amazonaws.com/lambda-docker-python:latest

Create Function

Either create it manually (the UI is very easy to figure out).

Or hacker-style 😎 in CLI. Creating via CLI requires a lambda role to be created beforehand (which I recommends anyway).

aws lambda create-function \
    --role arn:aws:iam::820792572713:role/fn_lambda_role \
    --function-name lambda-docker-python \
    --package-type Image \
    --code ImageUri=820792572713.dkr.ecr.us-west-2.amazonaws.com/lambda-docker-python:latest

Invoke Function

Enough set up, let's invoke our function in prod.

$ aws lambda invoke --function-name "lambda-docker-python" /dev/stdout

That's it!

Updating Function

Later on if we have any changes to our function we'd want to

Rebuild docker image
Re upload docker image
Update Function

aws lambda update-function-code \
    --function-name lambda-docker-python \
    --image-uri 820792572713.dkr.ecr.us-west-2.amazonaws.com/lambda-docker-python:latest

These are clearly can be automated. The script itself is left as exercise for the readers 😊 .

Find a very minimal sample code in github link below.