Skip to content
This repository has been archived by the owner on Dec 2, 2020. It is now read-only.

Commit

Permalink
Extract function deployment package from container image
Browse files Browse the repository at this point in the history
Bump version to 1.0, yay!
  • Loading branch information
clareliguori committed Oct 3, 2019
1 parent a9842a5 commit a3c8d94
Show file tree
Hide file tree
Showing 10 changed files with 211 additions and 69 deletions.
43 changes: 22 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,18 @@

[![Build Status](https://travis-ci.org/awslabs/aws-lambda-container-image-converter.svg?branch=master)](https://travis-ci.org/awslabs/aws-lambda-container-image-converter)

This container image converter tool (img2lambda) repackages container images (such as Docker images) into AWS Lambda layers, and publishes them as new layer versions to Lambda.
This container image converter tool (img2lambda) extracts an AWS Lambda function deployment package from a container image (such as a Docker image).
It also extracts AWS Lambda layers from a container image, and publishes them as new layer versions to Lambda.

The tool copies all files under '/opt' in the Docker image, maintaining the individual Docker image layers as individual Lambda layers. The published layer ARNs will be stored in a file 'output/layers.json', which can be used as input when creating Lambda functions. Each layer is named using a "namespace" prefix (like img2lambda or my-docker-image) and the SHA256 digest of the Docker image layer, in order to provide a way of tracking the provenance of the Lambda layer back to the Docker image that created it.
![img2lambda Demo](assets/demo.gif)

If a layer is already published to Lambda (same layer name, SHA256 digest, and size), it will not be published again. Instead the existing layer version ARN will be written to the output file.
To extract a Lambda function deployment package, the tool copies all files under '/var/task' in the container image into a deployment package zip file.

![img2lambda Demo](assets/demo.gif)
To extract Lambda layers, the tool copies all files under '/opt' in the container image, repackaging the individual container image layers as individual Lambda layer zip files.
The published layer ARNs are stored in a file 'output/layers.json', which can be used as input when creating Lambda functions.
Each layer is named using a "namespace" prefix (like 'img2lambda' or 'my-docker-image') and the SHA256 digest of the container image layer, in order to provide a way of tracking the provenance of the Lambda layer back to the container image that created it.
If a layer is already published to Lambda (same layer name, SHA256 digest, and size), it will not be published again.
Instead the existing layer version ARN will be written to the output file.

**Table of Contents**

Expand Down Expand Up @@ -41,7 +46,7 @@ GLOBAL OPTIONS:
--image-type value, -t value Type of the source container image. Valid values: 'docker' (Docker image from the local Docker daemon), 'oci' (OCI image archive at the given path and optional tag) (default: "docker")
--region value, -r value AWS region (default: "us-east-1")
--profile value, -p value AWS credentials profile. Credentials will default to the same chain as the AWS CLI: environment variables, default profile, container credentials, EC2 instance credentials
--output-directory value, -o value Destination directory for command output (default: "./output")
--output-directory value, -o value Destination directory for output: function deployment package (function.zip) and list of published layers (layers.json, layers.yaml) (default: "./output")
--layer-namespace value, -n value Prefix for the layers published to Lambda (default: "img2lambda")
--dry-run, -d Conduct a dry-run: Repackage the image, but only write the Lambda layers to local disk (do not publish to Lambda)
--description value, --desc value The description of this layer version (default: "created by img2lambda from image <name of the image>")
Expand Down Expand Up @@ -124,21 +129,21 @@ For example:

### Docker Example

Build the example Docker image to create a PHP Lambda custom runtime:
Build the example Docker image to create a PHP Lambda custom runtime and Hello World PHP function:
```
cd example
docker build -t lambda-php .
```

The example PHP functions are also built into the example image, so they can be run with Docker:
The Hello World function can be invoked locally by running the Docker image:
```
docker run lambda-php hello '{"name": "World"}'
docker run lambda-php goodbye '{"name": "World"}'
```

Run the tool to create and publish Lambda layers that contain the PHP custom runtime:
Run the tool to both create a Lambda deployment package that contains the Hello World PHP function, and to create and publish Lambda layers that contain the PHP custom runtime:
```
../bin/local/img2lambda -i lambda-php:latest -r us-east-1 -o ./output
```
Expand All @@ -154,26 +159,22 @@ podman build --format oci -t lambda-php .
podman push lambda-php oci-archive:./lambda-php-oci
```

Run the tool to create and publish Lambda layers that contain the PHP custom runtime:
Run the tool to both create a Lambda deployment package that contains the Hello World PHP function, and to create and publish Lambda layers that contain the PHP custom runtime:
```
../bin/local/img2lambda -i ./lambda-php-oci -t oci -r us-east-1 -o ./output
```

### Deploy Manually
Create a PHP function that uses the layers:
Create a PHP function that uses the layers and deployment package extracted from the container image:
```
cd function
zip hello.zip src/hello.php
aws lambda create-function \
--function-name php-example-hello \
--handler hello \
--zip-file fileb://./hello.zip \
--zip-file fileb://./output/function.zip \
--runtime provided \
--role "arn:aws:iam::XXXXXXXXXXXX:role/service-role/LambdaPhpExample" \
--region us-east-1 \
--layers file://../output/layers.json
--layers file://./output/layers.json
```

Finally, invoke the function:
Expand All @@ -191,18 +192,18 @@ cat hello-output.txt

### Deploy with AWS Serverless Application Model (SAM)

See [the sample template.yaml](example/function/template.yaml) and [the sample template.json](example/function/template.json).
See [the sample template.yaml](example/deploy/template.yaml) and [the sample template.json](example/deploy/template.json).

Insert the layers ARNs into the function definition:
```
cd function
cd example/deploy
sed -i 's/^- / - /' ../output/layers.yaml && \
sed -e "/LAYERS_PLACEHOLDER/r ../output/layers.yaml" -e "s///" template.yaml > template-with-layers.yaml
OR
cd function
cd example/deploy
sed -e "/\"LAYERS_PLACEHOLDER\"/r ../output/layers.json" -e "s///" template.json | jq . > template-with-layers.json
```
Expand Down Expand Up @@ -247,11 +248,11 @@ cat hello-output.txt

### Deploy with Serverless Framework

See [the sample serverless.yml](example/function/serverless.yml) for how to use the img2lambda-generated layers in your Serverless function.
See [the sample serverless.yml](example/deploy/serverless.yml) for how to use the img2lambda-generated layers in your Serverless function.

Deploy the function:
```
cd function
cd example/deploy
serverless deploy -v
```
Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.5.0
1.0.0
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,12 @@ functions:
hello:
handler: hello.hello
package:
include:
- src/hello.php
artifact: ../output/function.zip
layers:
${file(../output/layers.json)}
goodbye:
handler: goodbye.goodbye
package:
include:
- src/goodbye.php
artifact: ../output/function.zip
layers:
${file(../output/layers.json)}
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
"FunctionName": "sam-php-example-hello",
"Handler": "hello.hello",
"Runtime": "provided",
"CodeUri": "./",
"CodeUri": "../output/function.zip",
"Layers": "LAYERS_PLACEHOLDER"
}
},
Expand All @@ -18,7 +18,7 @@
"FunctionName": "sam-php-example-goodbye",
"Handler": "goodbye.goodbye",
"Runtime": "provided",
"CodeUri": "./",
"CodeUri": "../output/function.zip",
"Layers": "LAYERS_PLACEHOLDER"
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Resources:
FunctionName: sam-php-example-hello
Handler: hello.hello
Runtime: provided
CodeUri: ./
CodeUri: ../output/function.zip
Layers: LAYERS_PLACEHOLDER

Goodbye:
Expand All @@ -17,5 +17,5 @@ Resources:
FunctionName: sam-php-example-goodbye
Handler: goodbye.goodbye
Runtime: provided
CodeUri: ./
CodeUri: ../output/function.zip
Layers: LAYERS_PLACEHOLDER
15 changes: 10 additions & 5 deletions img2lambda/cli/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ func createApp() (*cli.App, *types.CmdOptions) {
app.EnableBashCompletion = true
app.Name = "img2lambda"
app.Version = version.VersionString()
app.Usage = "Repackages a container image into AWS Lambda layers and publishes them to Lambda"
app.Usage = "Repackages a container image into an AWS Lambda function deployment package. Extracts AWS Lambda layers from the image and publishes them to Lambda"
app.Action = func(c *cli.Context) error {
// parse and store the passed runtime list into the options object
opts.CompatibleRuntimes = c.StringSlice("cr")
Expand Down Expand Up @@ -56,7 +56,7 @@ func createApp() (*cli.App, *types.CmdOptions) {
},
cli.StringFlag{
Name: "output-directory, o",
Usage: "Destination directory for command output",
Usage: "Destination directory for output: function deployment package (function.zip) and list of published layers (layers.json, layers.yaml)",
Value: "./output",
Destination: &opts.OutputDir,
},
Expand Down Expand Up @@ -125,13 +125,18 @@ func repackImageAction(opts *types.CmdOptions, context *cli.Context) error {
imageLocation += ":latest"
}

layers, err := extract.RepackImage(imageLocation, opts.OutputDir)
layers, function, err := extract.RepackImage(imageLocation, opts.OutputDir)
if err != nil {
return err
}

if len(layers) == 0 {
return errors.New("No compatible layers found in the image (likely nothing found in /opt)")
if function.FileCount == 0 {
// remove empty zip file
os.Remove(function.File)
}

if len(layers) == 0 && function.FileCount == 0 {
return errors.New("No compatible layers or function files found in the image (likely nothing found in /opt and /var/task)")
}

if !opts.DryRun {
Expand Down
Loading

0 comments on commit a3c8d94

Please sign in to comment.