Code to generate and deploy the dashboard for the Cumulus API.
- Configuration
- Quick start
- Dashboard development
- Run the dashboard
- Deployment
- Testing
- Create a Dashboard Release
- Create a Backport Dashboard Release
Other pages:
The dashboard is populated from data retrieved from the Cumulus API. The environment for the Cumulus API must be predetermined and set before the dashboard can be built and deployed. The information needed to configure the dashboard is found in app/src/js/config/config.js, but it is generally preferred to set environmental variables overriding the default values during the build process.
Setting the following environment variables can override the default values.
| Env Name | Description | Default |
|---|---|---|
| APIROOT | The API URL. This must be set by the user. | example.com |
| AUTH_METHOD | The type of authorization method protecting the Cumulus API. [launchpad or earthdata] | earthdata |
| AWS_REGION | Region in which Cumulus API is running. | us-west-2 |
| DAAC_NAME | An identifier: e.g. LPDAAC, | Local |
| ENABLE_RECOVERY | If true, adds recovery options to the granule and collection pages. | false |
| HIDE_PDR | Whether to hide (or show) the PDR menu. | true |
| LABELS | Select daac localization. |
daac |
| STAGE | Identifier displayed at top of dashboard page: e.g. PROD, UAT | development |
| KIBANAROOT | <optional> Points to a Kibana endpoint. |
The dashboard source is available on github and can be cloned with git.
$ git clone https://github.com/nasa/cumulus-dashboardThe cloned directory ./cumulus-dashboard will be referred as the root directory of the project and commands that are referenced in this document, should start from that directory.
It is easy to build a production-ready, deployable version of the Cumulus dashboard without having to learn the complicated build process details. A single script, ./bin/build_dashboard_via_docker.sh, when combined with your dashboard's environment customizations, allows you to run the entire build process within a Docker container.
All of the environment variables in the configuration section are available to override with custom values for your dashboard. A recommended method is to store your variables in a source-able environment file for each dashboard you are going to build and deploy.
If you are using bash, export the values for each configuration option. An example production.env could look like:
# production.env
export APIROOT=https://afakeidentifier.cloudfront.net
export DAAC_NAME=MY-DAAC
export STAGE=production
export HIDE_PDR=falseAll values are optional except APIROOT which must point to the Cumulus API that the dashboard will connect to.
Set the environment and build the dashboard with these commands:
$ source production.env && ./bin/build_dashboard_via_docker.shThis script uses Docker Compose to build and copy the compiled dashboard into the ./dist directory. You can now deploy this directory to AWS behind CloudFront. If you are in NGAP, follow the instructions for "Request Public or Protected Access to the APIs and Dashboard" on the earthdata wiki page Using Cumulus with Private APIs.
You can also create a Docker container that will serve the dashboard behind a simple nginx configuration. Having a runnable Docker image is useful for testing a build before deployment or for NGAP Sandbox environments, where if you configure your computer to access Cumulus APIs via SSM, you can run the dashboard container locally against the live Sandbox Cumulus API.
The script ./bin/build_dashboard_image.sh will build a docker image containing the dashboard bundle served behind a basic nginx configuration. The script takes one optional parameter, the tag to name the generated image which defaults to cumulus-dashboard:latest. The same customizations as described in the previous section are available to configure your dashboard.
Example of building and running the project in Docker:
$ source production.env && ./bin/build_dashboard_image.sh cumulus-dashboard:production-1That command builds a Docker image with the name cumulus-dashboard and tag production-1. This image can be run in Docker to serve the Dashboard.
$ docker run --rm -it -p 3000:80 cumulus-dashboard:production-1In this example, the dashboard would be available at http://localhost:3000/ in any browser.
The dashboard uses node v20.12.2. To build/run the dashboard on your local machine, install nvm and run nvm install v20.12.2.
We use npm for local package management. To install the requirements:
$ nvm use
$ npm ciUse $ npm install when package.json is updated.
To build a dashboard bundle1:
$ nvm use
$ [SERVED_BY_CUMULUS_API=true] [DAAC_NAME=LPDAAC] [STAGE=production] [HIDE_PDR=false] [LABELS=daac] APIROOT=https://myapi.com npm run buildNOTE: Only the APIROOT environment variable is required and any of the environment variables currently set are passed to the build.
The compiled dashboard files (dashboard bundle) will be placed in the ./dist directory.
If you wish to serve the dashboard from behind CloudFront. Build a dist with your configuration including APIROOT and ensure the SERVED_BY_CUMULUS_API variable is unset. For NGAP users, follow the documentation to Request Public or Protected Access to the APIs and Dashboard, Step 5 of Cumulus Deployments in NGAP.
It is possible to serve the dashboard with the Cumulus API. If you need to do this, you must build the dashboard with the environment variable SERVED_BY_CUMULUS_API set to true. This configures the dashboard to work from the Cumulus dashboard endpoint. This option should only be considered when you can't serve the dashboard from behind CloudFront, for example in an NGAP Sandbox environment. NOTE: Your dashboard bucket must be in the bucket definitions in your Cumulus terraform.tfvars, otherwise you will not be able to access the bucket from the API.
cumulus-dashboard versions are distributed using tags in GitHub. You can select specific version in the following manner:
$ git clone https://github.com/nasa/cumulus-dashboard
$ cd cumulus-dashboard
$ git fetch origin ${tagNumber}:refs/tags/${tagNumber}
$ git checkout ${tagNumber}Then follow the steps noted above to build the dashboard locally or using Docker.
It is also possible to visit the repository at https://github.com/nasa/cumulus-dashboard/releases and download the source code bundle directly without cloning the repository.
During development you can run the webpack development webserver to serve the dashboard while you are developing. When you run the dashboard this way, the compiled code in ./dist is ignored, and the source code is served by the webpack-dev-server, which will watch for changes to the source and recompile as files are changed. Make sure you have installed the requirements and then:
APIROOT=http://<myapi>.com npm run serveThe dashboard should be available at http://localhost:3000
To run a built dashboard, first build the dashboard, then run:
$ npm run serve:prodThis runs a node http-server in front of whatever exists in the ./dist directory. It's fast, but will not pick up any changes as you are working.
First, build the dashboard. Then deploy the ./dist folder, the dashboard bundle, to an AWS bucket.
$ aws s3 sync dist s3://my-bucket-to-be-usedIf you are not in an NGAP environment, Look at the instructions for Hosting a static website on Amazon S3 and configuring a bucket as a static website.
Otherwise, follow the instructions for building and deploying the dashboard for cloudfront or the Cumulus API.
$ npm run testCommand to run a single test script locally using Ava:
$ npx ava test/../../test-name.jsFor the integration tests to work, you have to first run the localstack application, launch the localAPI and serve the dashboard first. Run the following commands in separate terminal sessions:
Run background localstack application.
$ npm run start-localstackServe the cumulus API (separate terminal)
$ npm run serve-apiServe the dashboard web application (another terminal)
$ [HIDE_PDR=false ENABLE_RECOVERY=true APIROOT=http://localhost:5001] npm run serveIf you're just testing dashboard code, you can generally run all of the above commands as a single docker-compose stack.
$ npm run start-dashboardThis brings up LocalStack, the Cumulus localAPI, and the dashboard.
Run the test suite (yet another terminal window)
$ npm run cypressWhen the cypress editor opens, click on run all specs.
For development and testing purposes only, you can run a Cumulus API locally. This requires docker compose in order to stand up the Docker containers that serve Cumulus API. There are a number of commands that will stand up different portions of the stack. See the Docker Service Diagram and examine the docker-compose*.yml file in the /localAPI/ directory to see all of the possible combinations. Described below are each of the provided commands for running the dashboard and Cumulus API locally.
Important Note: These docker compose commands do not build distributable containers, but are a provided as testing conveniences. The docker-compose[-*].yml files show that they work by linking your local directories into the container.
In order to run the Cumulus API locally you must first build the dashboard and then run the containers that provide LocalStack services.
These are started and stopped with the commands:
$ npm run start-localstack
$ npm run stop-localstackAfter these containers are running, you can start a cumulus API locally in a terminal window npm run serve-api, the dashboard in another window. [HIDE_PDR=false ENABLE_RECOVERY=true APIROOT=http://localhost:5001] npm run serve and finally cypress in a third window. npm run cypress.
Once the Docker app is running, If you would like to see sample data you can seed the database. This will load the same sample data into the application that is used during cypress testing.
$ npm run seed-databaseIf you prefer to stand up more of the stack in Docker containers, you can include the cumulus api in the docker-compose stack. To run the Cumulus API in a Docker container, (which still leaves running the dashboard and cypress up to you), just run the cumulusapi service.
The cumulusapi Docker service is started and stopped:
$ npm run start-cumulusapi
$ npm run stop-cumulusapithe start command, will exit successfully long before the stack is actually ready to run. The output looks like this:
> cumulus-dashboard@2.0.0 start-cumulusapi /Users/savoie/projects/cumulus/cumulus-dashboard
> docker compose -f ./localAPI/docker-compose.yml -f ./localAPI/docker-compose-serve-api.yml up -d
Creating localapi_shim_1 ... done
Creating localapi_localstack_1 ... done
Creating localapi_serve_api_1 ... doneIn order to find out that the stack is fully up and ready to receive requests, you can run the command npm run view-docker-logs to follow the progress of the stack. When the Docker logs have shown the following:
serve_api_1 | Starting server on port 5001and
localstack_1 | Ready.you should be able to verify access to the local Cumulus API at http://localhost:5001/token
Then you can run the dashboard locally (without Docker) [HIDE_PDR=false APIROOT=http://localhost:5001] npm run serve and open cypress tests npm run cypress.
The Docker compose stack also includes a command to let a developer start all development containers with a single command.
Bring up and down the entire stack (the localAPI and the dashboard) with:
$ npm run start-dashboard
$ npm run stop-dashboardThis runs everything, the backing Localstack container, the local Cumulus API and dashboard. Edits to your code will be reflected in the running dashboard. You can run cypress tests still with npm run cypress. As a warning, this command takes a very long time to start up because the containers come up in a specific order and generally this should be reserved for use by Earthdata Bamboo or some other continuous integration service. But, if you are using it locally, be sure to wait until all containers are fully up before trying to visit the dashboard which is exposed at http://localhost:3000
The stack is ready when the view-docker-logs task shows:
dashboard_1 | > NODE_ENV=production http-server dist -p 3000 --proxy http://localhost:3000?
dashboard_1 |
dashboard_1 | Starting up http-server, serving dist
dashboard_1 | Available on:
dashboard_1 | http://127.0.0.1:3000
dashboard_1 | http://172.18.0.2:3000
dashboard_1 | Unhandled requests will be served from: http://localhost:3000?
dashboard_1 | Hit CTRL-C to stop the serverIf something is not running correctly, or you're just interested, you can view the logs with a helper script, this will print out logs from each of the running docker containers.
$ npm run view-docker-logsThis can be helpful in debugging problems with the docker application.
A common error is running the dashboard containers when the cumulus core unit-test-stack is running on your machine. Just stop that stack and restart the dashboard stack to resolve.
ERROR: for localapi_shim_1 Cannot start service shim: driver failed programming external connectivity on endpoint localapi_shim_1 (7105603a4ff7fbb6f92211086f617bfab45d78cff47232793d152a244eb16feb): Bind for 0.0.0.0:9200 failed: port is already allocated
ERROR: for shim Cannot start service shim: driver failed programming external connectivity on endpoint localapi_shim_1 (7105603a4ff7fbb6f92211086f617bfab45d78cff47232793d152a244eb16feb): Bind for 0.0.0.0:9200 failed: port is already allocatedYou can run all of the cypress tests locally that Earthdata Bamboo runs with a single command:
$ npm run e2e-testsThis stands up the entire stack as well as begins the e2e service that will run all cypress commands and report an exit code for their success or failure. This is primarily used for CI, but can be useful to developers.
The master branch is the branch where the source code of HEAD always reflects the latest product release. The develop branch is the branch where the source code of HEAD always reflects the latest merged development changes for the next release. The develop branch is the branch where we should branch off.
When the source code in the develop branch reaches a stable point and is ready to be released, all of the changes should be merged back into master and then tagged with a release number.
We will make changes in the develop branch.
Create a new branch off of the develop branch for the release named release-vX.X.X (e.g. release-v1.3.0).
When changes are ready to be released, the version number must be updated in package.json.
See the minCompatibleApiVersion value in app/src/js/config/index.js.
Update the CHANGELOG.md. Put a header under the 'Unreleased' section with the new version number and the date.
Add a link reference for the GitHub "compare" view at the bottom of the CHANGELOG.md, following the existing pattern. This link reference should create a link in the CHANGELOG's release header to changes in the corresponding release.
Check to make sure there are Breaking Changes and All Changes section for the release if there are breaking changes. e.g. a new version of Cumulus API is required.
If this release corresponds to a Cumulus Core package release, update the version of @cumulus/api to the latest package version so that the integration tests will run against that version.
Test the dashboard against a live API deployed with the latest Cumulus packages. The dashboard should be served from an S3 bucket through the /dashboard API endpoint.
Create a PR for the release-vX.X.X branch against the develop branch. Verify that the Earthdata Bamboo CI build for the PR succeeds and then merge to develop.
Create a PR for the develop branch against the master branch. Verify that the Earthdata Bamboo CI build for the PR succeeds and then merge to master. Do not create a squash merge, but use a merge commit.
Push a new release tag to Github. The tag should be in the format v1.2.3, where 1.2.3 is the new version.
Create and push a new git tag:
$ git checkout master
$ git pull origin master
$ git tag -a v1.x.x -m "Release 1.x.x"
$ git push origin v1.x.xFollow the Github documentation to create a new release for the dashboard using the tag that you just pushed. Make sure to use the content from the CHANGELOG for this release as the description of the release on GitHub.
Create a PR for the master branch back into develop to bring the merge commit back into develop.
It is likely that no branch plan will exist for the master branch.
- In the Cumulus Dashboard in Bamboo (https://ci.earthdata.nasa.gov/browse/CUM-CDG), click
Actions -> Configure Planin the top right. - Next to
Plan branchclick the rightmost button that displaysCreate Plan Branchupon hover. - Click
Create plan branch manually. - Choose Branch Name
masterand then clickcreate. - Verify that the build has started for this plan.
git checkout vMAJOR.MINOR.PATCH
e.g.:
git checkout v12.0.1Create a new release branch off of the tag, and name the branch release-vMAJOR.MINOR.x (e.g. release-v12.0.x).
Note: If we already have the branch from prior backports, check out that branch instead of creating a new one.
git checkout -b release-vMAJOR.MINOR.x vMAJOR.MINOR.PATCH
e.g.:
git checkout -b release-v12.0.x v12.0.1 git cherry-pick [replace-with-commit-SHA]When changes are ready to be released, the version number must be updated in package.json.
Commit all changes and push the release branch (e.g. release-1.2.x) to GitHub.
Configure Bamboo to run automated tests against this branch by finding the branch plan for the release branch (release-1.2.x) and setting variables:
GIT_PR:true
See instruction in Create a Dashboard Release
Push a new release tag to Github. The tag should be in the format v1.2.3, where 1.2.3 is the new version. After the tag is created, you can create a new release from github.
Create and push a new git tag:
git tag -a v1.x.x -m "Release 1.x.x"
git push origin v1.x.xCreate a PR that merges ONLY the changelog updates back to develop.
1: A dashboard bundle is just a ready-to-deploy compiled version of the dashboard and environment.