How to automate one-time setup

[ilsaloving]

Hi all,

I am trying to create a single docker-compose file that deploys redash along with a database and a few other things as part of quick-to-deploy POC setup.

I would like to pre-provision redash with an admin user and sample schemas I'm preparing. Is there a recommended way of doing this?

[ilsaloving]

I'm seeing that it is possible to re-encrypt secrets. So maybe the simplest option is:

1. Do an initial installation
2. Setup the system as desired
3. Export the DB and store the current secret keys

Then on subsequent deployments:

1. Do the initial installation
2. Restore the DB from backup
3. Generate new keys and re-encrypt?

[justinclift]

Interesting. The concept itself sounds useful. 😄

As a thought with your POC deployment, is it for doing in Kubernetes?

If so, then the Redash helm stuff might be of use: https://github.com/getredash/contrib-helm-chart

[ilsaloving]

I will be doing k8s eventually, but for now this is just for running on a local machine or VM.

I am also adding Apache Airflow into the mix for ETL.

[justinclift]

As a 2nd thought, with this step:

Restore the DB from backup

Make sure you also backed up the environment file (env) as that contains the encryption key for the backup. Without that, the restore won't be 100% complete. 😉

[justinclift]

@ilsaloving This might be of interest, if you're still looking to pre-populate the Redash database with the initial admin user:

https://github.com/getredash/redash/wiki/How-to-add-the-first-user-(Admin)-to-Redash-without-using-the-web-interface

[ilsaloving]

That's certainly an option. I was also planning on exploring the API route, because direct DB changes risk problems if the schema ever changes. Either way, I got pulled in a different direction so this is currently on my back burner.

[justinclift]

No worries at all. 😄

[justinclift]

Hmmm, thinking about that a bit more, I'm not sure if there'd be a way to do it via API (as it currently stands anyway).

API calls are authenticated using API keys, and those are a "per user" thing. Without the initial/first user, there's no way API calls would have a key available to authenticate with.

In theory anyway. 😄

[eradman]

@ilsaloving here is an example of how the initial setup can be automated:

https://github.com/StarfishStorage/redash-email/blob/1b7ba5c99923e234b745c8b165028734e0d61d75/Makefile#L23-L25

[eradman]

It is true that the server needs to be running first, but if the commands are run in a script there is only a second between the time the database is created and the creation of the root account.

In production Redash is often behind a web server (such as Nginx), so the extra cautious approach is to stop the web server before initializing Redash.

Of course we could document both approaches!

[ilsaloving]

I think that sort of detail is beyond the scope of a setup script. If somebody wants redash to be public facing, then they need to factor that into their deployment. eg:

• temporarily run the server on a non-standard port that only the initialization script knows
• Create firewall rules that block general access, and release those rules once setup is complete
• If you're using a proxy, keep the proxy disabled until setup is complete.

[justinclift]

so the extra cautious approach is to stop the web server before initializing Redash.

Good point. Something like "don't enable the Nginx site for this Redash instance until the admin account creation commands have finished".

I'll have a go at updating the wiki page with words along those lines later on, after I've had some sleep. 😄

[justinclift]

I think that sort of detail is beyond the scope of a setup script.

Hmmm, I'd more go the other direction, and take a look at having the setup script try to do it safely.

Something along the lines of adding an optional extra parameter (or set of them) to be directly specified on the command line:

$ sudo ./setup.sh --create-admin --name "Foo Bar" --email "example@example.org"

... and if the --create-admin + name/email are provided then the setup script could disable port 500x, start the server container + populate the user info, then re-enable port 500x and start things up properly.

(Note there's no password option in the above example as using passwords directly on the command line is generally frowned upon. Command line arguments can be visible to other users on multi-user systems (ie ps -ef), so the password bit would probably need more thinking about)

[tinder-tder]

What we use is a similar setup (ec2 with docker) and we have the start script do all the bootstrapping

...
# ssm param with environ vars for config
get_param '/redash/config' >/opt/redash/env
chmod 0600 /opt/redash/env

#create db if doesnt exist
docker-compose -f /opt/redash/docker-compose.yml run --rm server create_db
#bootstrap admin access
ADMINS=$(get_param '/redash/admins')
for I in $ADMINS; do
	docker-compose -f /opt/redash/docker-compose.yml run --rm server manage users create_root $I $I --password $(cat /etc/machine-id) || true
done
docker-compose -f /opt/redash/docker-compose.yml -p redash up

password is whatever the machine id is, but we disable passwords in config since we use the SAML auth. and the instance can be ephemeral and autoreplaced/scaled as needed.

Edit: @ilsaloving
Sorry I should of mentioned we run a modified docker file, we use an external RDS, elasticcache, and alb instead of running those components in docker. All that is running is the docker server and workers