-
Notifications
You must be signed in to change notification settings - Fork 930
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: ✨ fargate-eventbridge-serverless pattern
- Loading branch information
Showing
7 changed files
with
1,255 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
FROM amd64/node:18.17-alpine | ||
|
||
COPY . . | ||
|
||
RUN yarn install | ||
|
||
CMD [ "handlers/handler.js" ] | ||
|
||
ENTRYPOINT [ "node" ] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,80 @@ | ||
# Leveraging Fargate for Scheduled Jobs | ||
|
||
This pattern contains a sample cloudformation template to deploy a scheduled AWS Fargate task using EventBridge running on an Amazon Elastic Container Service (ECS) cluster. The docker image is pushed to Amazon Elastic Container Registry (ECR) using cloudformation template without having to pre-push the image to ECR or another container library. The serverless framework plugin for fargate, `serverless-fargate` is used to deploy tasks on ECS Cluster. This task runs every 10th minute of the hour and a simple file containing JSON is put in S3 bucket. | ||
|
||
Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example. | ||
|
||
## Requirements | ||
|
||
* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources. | ||
* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured | ||
* [Git CLI](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) installed | ||
* [NodeJS](https://nodejs.org/en/download/) (LTS version) installed | ||
* [Serverless Framework CLI](https://www.serverless.com/framework/docs/getting-started) installed | ||
|
||
## Deployment Instructions | ||
|
||
1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository: | ||
|
||
``` sh | ||
git clone https://github.com/aws-samples/serverless-patterns | ||
``` | ||
|
||
1. Change directory to the fargate-eventbridge pattern directory: | ||
|
||
``` sh | ||
cd serverless-patterns/fargate-eventbridge-serverless | ||
``` | ||
|
||
1. Replace the subnet and security group ids in serverless.yml file to your default ids. | ||
|
||
1. From the command line, use Serverless Framework to deploy the AWS resources for the pattern as specified in the serverless.yml file: | ||
|
||
``` sh | ||
serverless deploy --verbose | ||
``` | ||
|
||
The above command will deploy resources to `ap-south-1` region by default. You can override the target region with `--region <region>` CLI option, e.g. | ||
|
||
``` sh | ||
serverless deploy --verbose --region eu-west-1 | ||
``` | ||
|
||
## How it works | ||
|
||
- The image is constructed directly from the Dockerfile that is provided. | ||
- The image is pushed to Amazon Elastic Container Repository (ECR). | ||
- Fargate Task role is created. | ||
- Fargate Execution role is created. | ||
- The S3 bucket for output is created. | ||
- The ECS cluster is created. | ||
- The Task Definition is created. This also passes the environment variable to the scheduled task. | ||
- Finally EventBridge Rule is created for running task every 10th minute of the hour. | ||
|
||
## Testing | ||
|
||
To test the deployment, you can verify by going to the output S3 bucket and find the output JSON file which contains a message and timestamp as below: | ||
``` | ||
{ | ||
"message": "Task executed from Fargate Cluster.", | ||
"timestamp": 1692648300534 | ||
} | ||
``` | ||
|
||
|
||
## Cleanup | ||
|
||
1. Empty the output S3 bucket. | ||
|
||
1. Delete the stack: | ||
|
||
```sh | ||
serverless remove --verbose | ||
``` | ||
|
||
NOTE: You might need to add `--region <region>` option to AWS CLI command if you AWS CLI default region does not match the one, that you used for the Serverless Framework deployment. | ||
|
||
---- | ||
Copyright 2023 Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||
|
||
SPDX-License-Identifier: MIT-0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,64 @@ | ||
{ | ||
"title": "Leveraging Fargate for Scheduled Jobs", | ||
"description": "Create a scheduled AWS Fargate Task running on AWS ECS Cluster and EventBridge Rule using Serverless Framework", | ||
"language": "Node.js", | ||
"architectureURL": "", | ||
"videoId": "", | ||
"level": "100", | ||
"framework": "Serverless Framework", | ||
"introBox": { | ||
"headline": "How it works", | ||
"text": [ | ||
"This pattern deploys an AWS Fargate Task on AWS ECS Cluster which runs on EventBridge Rule.", | ||
"", | ||
"The docker image for the task based on Node.js is built first, then the image is pushed to AWS ECR. AWS ECS pulls the image from ECR and runs the image on Fargate. A serverless plugin called 'serverless-fargate' is used to create the template. The job is scheduled to run on every 10th minute of the hour.", | ||
"", | ||
"A simple JSON file containing message string and timestamp, is pushed to S3 bucket. All the necessary resources and permissions are configured using serverless framework." | ||
] | ||
}, | ||
"gitHub": { | ||
"template": { | ||
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/fargate-eventbridge-serverless", | ||
"templateURL": "serverless-patterns/fargate-eventbridge-serverless", | ||
"projectFolder": "fargate-eventbridge-serverless", | ||
"templateFile": "serverless.yml" | ||
} | ||
}, | ||
"resources": { | ||
"headline": "Additional resources", | ||
"bullets": [ | ||
{ | ||
"text": "Creating Amazon ECS resources with AWS CloudFormation", | ||
"link": "https://docs.aws.amazon.com/AmazonECS/latest/userguide/creating-resources-with-cloudformation.html" | ||
}, | ||
{ | ||
"text": "Serverless Fargate Plugin", | ||
"link": "https://www.serverless.com/plugins/serverless-fargate" | ||
} | ||
] | ||
}, | ||
"deploy": { | ||
"text": [ | ||
"<code>serverless deploy --verbose</code>" | ||
] | ||
}, | ||
"testing": { | ||
"text": [ | ||
"See the Github repo for detailed testing instructions." | ||
] | ||
}, | ||
"cleanup": { | ||
"text": [ | ||
"<code>serverless remove --verbose</code>." | ||
] | ||
}, | ||
"authors": [ | ||
{ | ||
"headline": "Presented by Tejas Dhamecha, Software Engineer at Distinction-Dev, India.", | ||
"name": "Tejas Dhamecha", | ||
"bio": "Tejas Dhamecha is a Software Engineer working at Distinction-Dev, India who has a passion for building serverless solutions.", | ||
"linkedin":"https://www.linkedin.com/in/tejas-dhamecha/", | ||
"imageURL": "https://0.gravatar.com/avatar/98a4c425988ef7ee758a28ba98b947b7860c3182e898340b7b3c12bc441effb7?size=256" | ||
} | ||
] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
const { PutObjectCommand, S3Client } = require("@aws-sdk/client-s3"); | ||
const s3 = new S3Client({}); | ||
|
||
const handler = async () => { | ||
const timestamp = Date.now() | ||
const message = JSON.stringify({ | ||
message: 'Task executed from Fargate Cluster!', | ||
timestamp | ||
}) | ||
const putCommand = new PutObjectCommand({ | ||
Bucket: process.env.DESTINATION_BUCKET, | ||
Key: timestamp + '.json', | ||
ContentType: 'application/json', | ||
Body: message, | ||
}); | ||
const result = await s3.send(putCommand); | ||
console.log('result: ', result) | ||
}; | ||
|
||
handler() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
{ | ||
"name": "fargate-eventbridge-serverless", | ||
"version": "1.0.0", | ||
"main": "index.js", | ||
"license": "MIT", | ||
"devDependencies": { | ||
"serverless-fargate": "^0.0.12" | ||
}, | ||
"dependencies": { | ||
"@aws-sdk/client-s3": "^3.395.0" | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
service: fargate-eventbridge-serverless | ||
frameworkVersion: '3' | ||
provider: | ||
name: aws | ||
stage: ${opt:stage, 'dev'} | ||
region: ${opt:region, 'ap-south-1'} | ||
iamRoleStatements: | ||
- Effect: Allow | ||
Action: | ||
- s3:CreateBucket | ||
Resource: arn:aws:s3:::${self:custom.destinationBucket}/* | ||
ecr: | ||
images: | ||
hello-task: | ||
path: ./ | ||
file: Dockerfile | ||
|
||
custom: | ||
destinationBucket: ${self:service}-${self:provider.stage}-destination-bucket | ||
|
||
fargate: | ||
clusterName: fargate-eventbridge-serverless | ||
containerInsights: true | ||
memory: '0.5GB' | ||
cpu: 256 | ||
architecture: X86_64 | ||
vpc: | ||
assignPublicIp: true | ||
securityGroupIds: | ||
- sg-1234 | ||
subnetIds: | ||
- subnet-1234 | ||
iamRoleStatements: | ||
- Effect: Allow | ||
Action: | ||
- s3:PutObject | ||
Resource: arn:aws:s3:::${self:custom.destinationBucket}/* | ||
|
||
tasks: | ||
hello-task: | ||
name: hello-task | ||
image: hello-task | ||
vpc: | ||
assignPublicIp: true | ||
securityGroupIds: | ||
- sg-1234 | ||
subnetIds: | ||
- subnet-1234 | ||
memory: '0.5GB' | ||
cpu: 256 | ||
schedule: cron(0/10 * * * ? *) | ||
environment: | ||
DESTINATION_BUCKET: ${self:custom.destinationBucket} | ||
|
||
resources: | ||
Resources: | ||
DestinationBucket: | ||
Type: AWS::S3::Bucket | ||
Properties: | ||
BucketName: ${self:custom.destinationBucket} | ||
|
||
plugins: | ||
- serverless-fargate |
Oops, something went wrong.