Setting up a dev environment: step by step tutorial
This guide will install OAE locally for a development setup. Node is required on the host machine whereas the remaining servers will run as docker containers.
The recommended way to install docker is to follow the official guide at https://docs.docker.com/engine/installation/. Make sure you have docker version >= 17.x and docker-compose version >= 1.6.0 before you proceed to cloning the repos. Check your versions by running the following commands:
$ docker -v
Docker version 17.03.0-ce, build 60ccb2265
$ docker-compose -v
docker-compose version 1.11.2, build dfed245
Also, don't forget the post-install instructions if you're using linux.
git clone https://github.com/oaeproject/Hilary.git && cd Hilary
git submodule init
git submodule update
# Next step is optional
cd 3akai-ux && git checkout master # because HEAD is detached after pulling submodules by default
If you accept the following directory structure, docker-compose will work out of the box.
<some-local-path>
|-- Hilary
|-- 3akai-ux
|-- tmp
|-- previews
|-- uploads
|-- files
|-- data
|-- elasticsearch
|-- cassandra
If you want to use different (local) paths, make sure to change container volumes accordingly on docker-compose.yml:
# this will build the hilary:latest image and create all containers
docker-compose up --no-start --build oae-cassandra oae-redis oae-rabbitmq oae-elasticsearch oae-hilary
NOTE: if the previous step fails due to network problems, try changing the DNS server to Google's: 8.8.8.8 or 8.8.4.4. In order to do this, either use your operating system's settings or do it via the command line interface by editing /etc/resolv.conf and making sure these two lines are on top:
nameserver 8.8.8.8
nameserver 8.8.4.4
In order to install dependencies for the frontend and the backend, we need lerna installed in order to run a one-off command for each:
# on Hilary folder
./prepare-etherpad.sh
cd ethercalc && npm install
# then the same for 3akai-ux
cd 3akai-ux && npm install
If we're looking to use HTTPS via nginx, first we need to create the SSL certificate. You can do do that by running:
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout 3akai-ux/nginx/nginx-selfsigned.key -out 3akai-ux/nginx/nginx-selfsigned.crt
This will create two new files: 3akai-ux/nginx/nginx-selfsigned.key and 3akai-ux/nginx/nginx-selfsigned.crt.
Then, run the following command:
openssl dhparam -out 3akai-ux/nginx/dhparam.pem 2048
This may take a few minutes, but when it's done you will have the file nginx/dhparam.pem that you can use in your configuration.
--
Before moving on to the next step, make sure these three files exist otherwise there will be errors.
Open the file nginx.docker.conf and make sure these lines:
...
server oae-hilary:2000;
...
server oae-hilary:2001;
...
..become:
...
server 172.20.0.1:2000; # `172.20.0.1` is the IP address of the host machine, which can be obtained by running `/sbin/ip route|awk '/default/ { print $3 }'` from any container (e.g. `docker exec -it oae-nginx sh`).
...
server 172.20.0.1:2001;
...
If you're using mac osx, you'll need to use the external IP address (e.g. en0) instead of the docker0 IP address for oae-nginx to access Hilary, like this:
...
server 192.168.1.2:2000; # assuming 192.168.1.2 is the external network IP address
...
server 192.168.1.2:2001; # assuming 192.168.1.2 is the external network IP address
...
Before running the app we need to ensure the schema exists on the database. To achieve that we need to run
npm run migrate
If the database settings are correct (config.js) then the output should resemble the following:
Now run docker-compose up -d oae-cassandra oae-redis oae-elasticsearch oae-nginx and then docker-compose logs -f to check the logs for all containers.
You may then run npm run servers locally on the terminal to start the server, ethercalc and etherpad
In order to set up twitter authentication, you'll need to set your twitter dev account environment variables like this:
export TWITTER_KEY="<your key here>"
export TWITTER_SECRET"="<your secret here>"
Same thing goes for google auth and facebook auth. The environment variables for each are:
export GOOGLE_CLIENT_ID=""
export GOOGLE_CLIENT_SECRET=""
export FACEBOOK_APP_ID=""
export FACEBOOK_APP_SECRET=""
This is enough to run all the tests locally in a dev environment. For production purposes, all environment variables can and should be overwritten by the admin in the tenant configuration form.
OAE is a multi-tenant system that discriminates the tenant by the host name with which you are accessing the server. In order to support the "Global Tenant" (i.e., the tenant that hosts the administration UI) and a "User Tenant", you will need to have at least 2 different host names that point to your server. To do this, you will need to add the following entries to your /etc/hosts file:
127.0.0.1 admin.oae.com
127.0.0.1 tenant1.oae.com
Where admin.oae.com is the hostname that we will use to access the global administration tenant and tenant1.oae.com would be one of many potential user tenant hosts. After making this change, you should now be able to visit http://admin.oae.com \o/
This same DNS information must be made explicit in the docker-compose.yml file, to make sure that the oae-hilary container can connect to the oae-nginx container holding HTTP server (for instance, for preview processing purposes). Go to the file and look for the following section:
extra_hosts:
- "admin.oae.com:172.20.0.9"
- "tenant1.oae.com:172.20.0.9"
As you see, we already included both admin.oae.com and tenant1.oae.com, both associated with the oae-nginx static IP. If you're looking to add an extra host, say, tenant2.oae.com, then you should add the following line and you're good to go:
- "tenant2.oae.com:172.20.0.9"
When you start the server, all data schemas will be created for you if they don't already exist. A global administrator user and global administration tenant will be ready for you as well. You can use these to create a new user tenant that hosts the actual OAE user interface.
- Visit http://admin.oae.com/ (substitute "admin.oae.com" with the administration host you configured in
/etc/hosts) - Log in with username and password:
administrator/administrator - Click the "Tenants" header to open up the actions
- Click "Create tenant"
- Choose an alias (a short, unique 2-5 character alphanumeric string such as "oae"), and a name of your liking.
- For the Host field, use the host you configured for your user tenant in
/etc/hosts(e.g., "tenant1.oae.com") - Click "Create new tenant"
You can now access the user tenant by their host http://tenant1.oae.com and start creating new users.
To create a new user, use either the Sign Up link at the top left, or the Sign In link at the top right.
Tip: OAE requires that users have an email address that is verified VIA an email that is sent to the user. To avoid the requirement of having a valid email server configuration, you can instead watch the app server logs when a user is created or their email address is updated. When config.email.debug is set to true in config.js, the content of the verification email can be seen in the logs, and you can copy/paste the email verification link from the log to your browser to verify your email. The URL will look similar to: http://tenant1.oae.com/?verifyEmail=abc123
We're looking forward to seeing your contributions to the OAE project!
To run tests just make sure you have installed all dependencies (npm install) and run npm run test. To run tests on a specific module, just append its path as follows: npm run test-modules packages/oae-principals/tests.