Add another CDK app to test synth and deploy

cloud/README.md

@@ -18,43 +18,73 @@ This should be much easier, as it is natively supported by API Gateway, but it s
 
 ## Commands
 
+The pipeline only needs to be synthesized and deployed once, after which it will self-update whenever triggered. It is
+currently configured to run on merging to `main` branch, so adjust that if you want to trigger on a different branch.
+
 `npm run cdk:synth` 
-Generates the CloudFormation templates for a build pipeline - dev stage.
+Generates the CloudFormation templates for a "dev stage" build pipeline.
 
 `npm run cdk:synth:prod` 
-Generates the CloudFormation templates for a build pipeline - production stage.
+Generates the CloudFormation templates for a "production stage" build pipeline.
 
 `npm run cdk:synth -- --context STAGE={STAGENAME}` 
-Generates the CloudFormation templates for a build pipeline - stage given by `{STAGENAME}`.
+Generates the CloudFormation templates for a build pipeline, stage given by `{STAGENAME}`.
 
 `npm run cdk:deploy:all` 
 Deploys the synthesized pipeline to an AWS Environment, as defined by your active AWS config profile.
 
-Note that the pipeline only needs to be deployed once, after which it self-updates whenever it is triggered. It is
-currently configured to run on merging to `main` branch, so adapt to your needs.
-
 `npm run cdk:destroy:all` 
 Destroys the deployed pipeline in your remote AWS Environment.
 
-Destroying the pipeline stack does not destroy the application stacks, so those will need to be deleted manually in the
-AWS Console.
+Note that destroying the pipeline stack does not destroy the application stacks deployed by the pipeline, so those would
+need to be deleted manually in the AWS Console.
+
+## Testing stack changes
+
+As the pipeline deploys the application stacks, it is wise to test any changes to those stacks before committing them.
+You can do this by synthesizing just the application stacks locally, and deploying to AWS as `dev` stage.
 
+There is one small task to complete before you begin. In AWS Secrets Manager, you will find a secret storing API key and
+secret values for the `prod` stage, which the server needs for successful startup. You must create a new secret for the
+dev stage, with the same OPENAI_API_KEY value and any random string for SESSION_SECRET. Once that is in place, you can
+synthesize and deploy the stacks for testing. Once you have finished, please delete the secret to avoid unnecessary
+costs.
 
-Additionally, 
+`npm run cdk:test:synth` - synthesizes just the application stacks (i.e. not the pipeline)
+
+`npm run cdk:test:deploy` - deploys these stacks to AWS as "dev" stage
+
+All being successful, you should see the application login screen at `https://dev.spylogic.ai`. Log into the AWS Console to add a
+user to the dev Cognito userpool, then log into the UI to test app deployment was successful.
+
+`npm run cdk:test:destroy` - Remember to destroy the stacks after testing, else you will rack up costs!
 
 ---
 
+## A note on costs
+
+At the time of writing, current infrastructure costs us around $60 per month, with just two AZs for the load balancer,
+deployed into `eu-north-1`. This is one of the [greenest AWS regions](https://app.electricitymaps.com/map), but costs
+are about average. The vast majority of the bill is for the VPC, Load Balancer and NAT EC2 Instance. We have tasks on
+our todo list to reduce these costs (such as removing the NAT Instance in favour of IPv6 egress), but those are
+work-in-progress.
+
+The bottom line: remember to destroy your stacks when no longer needed!
+
 ## First-time admin tasks
 
-If you are setting up the CDK project for the first time, there are a few setup tasks you must complete.
+When setting up the CDK project for the first time, there are a few one-time tasks you must complete.
 
 ### Bootstrapping the CDK using a Developer Policy
 
 In order to deploy AWS resources to a remote environment using CDK, you must first
 [bootstrap the CDK](https://docs.aws.amazon.com/cdk/v2/guide/bootstrapping.html). For this project, as per
-[CDK guidelines](https://aws.amazon.com/blogs/devops/secure-cdk-deployments-with-iam-permission-boundaries/), we use a
-lightweight permissions boundary to restrict permissions, to prevent creation of new users or roles with elevated
-permissions. See `cdk-developer-policy.yaml` for details.
+[CDK guidelines](https://aws.amazon.com/blogs/devops/secure-cdk-deployments-with-iam-permission-boundaries/), we are
+using a lightweight permissions boundary to restrict permissions, to prevent creation of new users or roles with
+elevated permissions. See `cdk-developer-policy.yaml` for details.
+
+Note that once the pipeline is deployed, it is a good idea to restrict permissions further, so that only the pipeline
+can make changes to the stacks.
 
 Create the permissions boundary CloudFormation stack:
 
@@ -75,9 +105,20 @@ npm install
 npx cdk bootstrap --custom-permissions-boundary cdk-developer-policy
 ```
 
-Unless your default region is `us-east-1`, you will also need to bootstrap that region, as certificates for CloudFront
-currently need to be deployed into that region:
+Unless your default region is `us-east-1`, you will also need to bootstrap this region, as certificates for CloudFront
+currently need to be deployed there:
 
 ```
 npx cdk bootstrap --custom-permissions-boundary cdk-developer-policy aws://YOUR_ACCOUNT_NUMBER/us-east-1
 ```
+
+### Server secrets
+
+The Node Express server needs a couple of secret values, which are injected into the container environment during
+deployment via AWS Secrets Manager.
+
+You will need to create a secret in your chosen region named `{stagename}/SpyLogic/ApiKey`, where stagename matches the
+name used during Synth (for example, "dev" or "prod"). Within this secret, you will need values for OPENAI_API_KEY and
+SESSION_SECRET.
+
+Refer to the [main README](../README.md) for further details on these.

cloud/bin/application.ts

@@ -0,0 +1,89 @@
+#!/usr/bin/env node
+import { App, Environment } from 'aws-cdk-lib/core';
+import 'source-map-support/register';
+
+import {
+ appName,
+ resourceDescription,
+ resourceId,
+ stackName,
+ stageName,
+ ApiStack,
+ AuthStack,
+ CertificateStack,
+ HostedZoneStack,
+ UiStack,
+} from '../lib';
+
+/*
+This application can be used to test stack changes before they are deployed by
+the production pipeline. See the cloud README for details.
+*/
+
+const app = new App();
+const generateStackName = stackName(app);
+const generateDescription = resourceDescription(app);
+
+/* Common stack resources */
+
+// NOTE: Need DOMAIN_NAME and HOSTED_ZONE_ID env vars, see "cloud/.env.example"
+// If you have access to SpyLogic AWS account, these are viewable in Parameter Store.
+const env: Environment = {
+ account: process.env.CDK_DEFAULT_ACCOUNT,
+ region: process.env.CDK_DEFAULT_REGION,
+};
+
+const tags = {
+ owner: appName,
+ stage: stageName(app),
+};
+
+/* Stack constructs */
+
+const hostedZoneStack = new HostedZoneStack(app, generateStackName('hostedzone'), {
+ description: generateDescription('Hosted Zone stack'),
+ env,
+ tags,
+});
+
+const certificateStack = new CertificateStack(app, generateStackName('certificate'), {
+ description: generateDescription('Certificate stack'),
+ env,
+ tags,
+ domainName: hostedZoneStack.topLevelDomain.value as string,
+ hostedZone: hostedZoneStack.hostedZone,
+});
+
+const authStack = new AuthStack(app, generateStackName('auth'), {
+ description: generateDescription('Auth stack'),
+ env,
+ tags,
+ domainName: hostedZoneStack.topLevelDomain.value as string,
+});
+
+new ApiStack(app, generateStackName('api'), {
+ description: generateDescription('API stack'),
+ env,
+ tags,
+ apiDomainName: certificateStack.apiDomainName,
+ certificate: certificateStack.loadBalancerCert,
+ customAuthHeaderName: authStack.customAuthHeaderName,
+ customAuthHeaderValue: authStack.customAuthHeaderValue,
+ domainName: hostedZoneStack.topLevelDomain.value as string,
+ hostedZone: hostedZoneStack.hostedZone,
+});
+
+new UiStack(app, generateStackName('ui'), {
+ description: generateDescription('UI stack'),
+ env,
+ tags,
+ apiDomainName: certificateStack.apiDomainName,
+ certificate: certificateStack.cloudFrontCert,
+ customAuthHeaderName: authStack.customAuthHeaderName,
+ customAuthHeaderValue: authStack.customAuthHeaderValue,
+ domainName: hostedZoneStack.topLevelDomain.value as string,
+ hostedZone: hostedZoneStack.hostedZone,
+ hostBucketName: resourceId(app)('host-bucket'),
+ parameterNameUserPoolClient: authStack.parameterNameUserPoolClient,
+ parameterNameUserPoolId: authStack.parameterNameUserPoolId,
+});

cloud/cdk.json

@@ -1,5 +1,5 @@
 {
- "app": "npx ts-node --prefer-ts-exts -r dotenv/config bin/cloud.ts",
+ "app": "npx ts-node --prefer-ts-exts -r dotenv/config bin/pipeline.ts",
  "watch": {
  "include": ["**"],
  "exclude": [

cloud/lib/index.ts

@@ -1,3 +1,10 @@
 export * from './resourceNamingUtils';
+
+export { ApiStack } from './api-stack';
+export { AuthStack } from './auth-stack';
+export { CertificateStack } from './certificate-stack';
+export { HostedZoneStack } from './hostedzone-stack';
+export { UiStack } from './ui-stack';
+
 export { PipelineAssistUsEast1Stack } from './pipeline-assist-useast1-stack';
 export { PipelineStack } from './pipeline-stack';

cloud/package.json

@@ -2,7 +2,7 @@
  "name": "cloud",
  "version": "1.0.3",
  "bin": {
- "cloud": "bin/cloud.js"
+ "cloud": "bin/pipeline.js"
  },
  "scripts": {
  "cdk:synth": "cdk synth -q \"*\"",
@@ -13,6 +13,10 @@
  "cdk:destroy": "cdk destroy --app cdk.out",
  "cdk:destroy:all": "cdk destroy --app cdk.out --all",
  "cdk:clean": "rimraf cdk.out",
+ "cdk:test:synth": "cdk synth -o cdk.test.out -a \"npx ts-node --prefer-ts-exts -r dotenv/config bin/application.ts\"",
+ "cdk:test:deploy": "cdk deploy --app cdk.test.out --all",
+ "cdk:test:destroy": "cdk destroy --app cdk.test.out --all",
+ "cdk:test:clean": "rimraf cdk.test.out",
  "codecheck": "concurrently \"npm run lint:check\" \"npm run format:check\"",
  "format": "prettier . --write",
  "format:check": "prettier . --check",