The Redis playground is based on CloudFormation, the AWS Infastracture as software tool. The following guidelines are meant for people with zero experience in CloudFormation tool.
If you are familiar with the concept of stacks and lambdas you can follow the short version. Otherwisse go Ahead for the detailed one.
Below you can find a schema of the architecture of what we are going to build:
- Create a stack using the cloud-formation/infrastacture/infrastructure.yaml and name it redis-demo-infra
- Before creating the lambdas:
- download the zips in the lambda-zips directory
- smarthome-producer.yaml
- smarthome-consumer.yaml
- download the dataset in the dataset directory
- load dataset and lambdas in s3
- update the cloud-formation/use-cases/smarthome.yaml in the mapping section at the top of the file to reflect your s3 bucket and key
- download the zips in the lambda-zips directory
- Create another stack using the updated cloud-formation/use-cases/smarthome.yaml
- Once you have the producer lambda update the link to your s3 bucket
- Run the lambdas
Table of Contents
- How to install the Redis Playground
- AWS account
- s3 bucket created (no need for public access)
- if you need guidance on how to create a bucket
- Clone this repo
git clone https://github.com/DataReply/redis-streaming-playground.git
Login to the AWS Console. In the Top left corner click the search bar close 9 dots in a square shape. Type CloudFormation and click the first entry
In the main menu of CloudFormation click Create Stack. In the next window click With new Resources
Leave the first section Prerequisite - Prepare template as it is with the option Template is ready selected. In the Specify Template section select Upload a template file Click Choose file and from this repo load the file called infrastructure.yaml in the cloud-formation/infrastructure folder. Click Next
In the next window choose a name of your choice, no other information is needed.
Add tags if needed otherwise click next.
Scroll to bottom, acknowledge the terms of AWS. Click create stack and wait for the resources to be created.
Please Note that the OpenSearch server need 15-20 min to be instanciated.
In this section we are going to deploy the lambdas using CloudFormation, we will select the smarthome use case. If at the moment you are reading this is the only usecase please don't panic, we will add more in the future.
Download the lambdas in the lamda-zips folder. You will have two zips for each use case: one for the consumer and one for the producer. You can find them here:
- lambda-zips/smarthome-consumer.zip
- lambda-zips/smarthome-producer.zip
Load both the zips in s3. If you need guidance on how to load file into your s3 bucket follow this guide.
Change the mapping in the file cloud-formation/use-cases/smarthome.yaml to and set your s3 bucket name and path to the downloaded consumer and producer zip.
Repeat the steps from 1.1 to 1.4 to deploy a new CloudFormation stack which will create the producer and consumer Lambdas. This time you should load the cloud-formation/use-cases/smarthome.yaml
This is not an operative section, you don't need to do anything, relax. Let's recap the resources you have created so far:
- Network related resources (VPC, subnet, route tables, associations...)
- Permission related resources (security group, roles)
- ElastiCache cluster, OpensSearchServer
- Consumer and Producer Lambdas
If more help is needed check the CloudFormation doc.
From the use cases stack select resources click the producer lambda, in this case SmartHomeProducer
Click and edit the general configuration. Set 5min timeout (worst case scenario).
Click test. If it is the first time don't worry about the input, just give a random name like 'test' and leave anything else as it is. Clik Run.
Now run the code and see that the first records are produced. You should see something like this:
Do the same with the SmartHomeConsumer.
This part is optional and can be skipped. It explains instruction to get a graph from the dataset.
Go to the CloudFormation page in AWS. From there select the infrastracture stack. Select the resources and then click the OpenSearch link.
Click the Open Search Dashboard URL
Log in with the credentials you find in the infrastructure.yaml
- look for the OpenSearch section towards the bottom
- look for the MasterUserOptions property and right below you will find hardcoded username and password
Requirement: Please note that to successfully create an index you need to load at least some data into OpenSearch. It is sufficient to run at least once the consumer lambda.
One you logged in close the discover tab and open the menu on the left. Click Stack Management in the Management block. Click index Patterns in the menu on the left. Click Create index pattern.
Provide any index-pattern that would match the index called 'first-index' created by the consumer.
The output should look like this:
Once the data is loaded you can create a visualization of your choice, if you need guidance you can always take a look to the doc