This project demonstrates how to meter a lambda function.
Amberflo provides different methods to ingest meter records:
- Amberflo API call, or
- Adding a file to a designated S3 bucket, or
- Sending a message to a designated SQS queue.
In the context of a running service, in our case an AWS Lambda function, this can be done:
- during the lambda execution itself, or
- by subscribing to the lambda logs, or
- (TODO) from a lambda extension.
In this project, you can see how to implement each of these methods.
This project uses AWS SAM to create an API that will be served by a lambda function. When this function runs, it causes a meter record to be ingested.
Each ingestion method is triggered by a different call to the API. For instance, POST $API_URL/direct-s3 will cause the ingestion to happen immediately via your designated S3 bucket.
In the direct ingestion methods, the ingestion happens in your main lambda code.
-
- in this case, you call the Amberflo API directly from your lambda to send the meter records.
-
- in this case, you use the official AWS SDK to add your meter records as files to an Amberflo provided S3 bucket.
-
- in this case, you use the official AWS SDK to send your meter records as messages to an Amberflo provided SQS queue.
All these methods support the same payload format.
These methods are easier to setup because they require no additional infrastructure, but they will make an HTTP request to the underlying service, so you'll need to handle errors. They'll also increase the running time of your lambda and may require installing additional dependencies. For these reasons, an indirect method is preferable.
In the indirect ingestion methods, your meter records are logged by the main lambda, and another lambda processes the log records, extracts the meter records and ingests them.
This facilitates error handling (you can always retry processing the logs) and keeps your lambda code simple. Although the infrastructure cost is higher (the log subscription and consumer), it does not grow with each new lambda you need to meter but remains constant.
- with CloudWatch subscriber Lambda
- in this case, the ingestion lambda subscribes directly to your CloudWatch log group
- with Kinesis subscriber Lambda
- in this case, you send your CloudWatch logs to a Kinesis stream, and your ingestion lambda consumes the stream
Currently, a CloudWatch log group can have at most 2 subscription filters, so having Kinesis intermediate the ingestion may be useful if you also need the logs for other purposes (i.e. Kinesis allows fanning out).
To read more about the CloudWatch integration, see this doc.
To run this example you'll need:
- An Amberflo account
- A meter and a customer in Amberflo
- An AWS account
- AWS CLI installed
- AWS SAM installed
- Node.js
Fill in the lambda environment variables in the SAM template file and run the deploy script.
When the deployment is complete, find the API stage URL in the API Gateway console and set it as the api_url in the test script.
Now you can run the test script to cause the lambda to execute and meter events ingested.
In the Amberflo UI, you should now be able to see the usage.