environment.adoc

Environment

Database

The Model Catalogue is currently tested and deployed against a MySQL database server, which uses InnoDB storage engine. The application was tested with versions 5.6.x and 5.7.x. The database schema is created automatically by Grails. It is highly recommended to add the following indexes (for some reason):

CREATE INDEX change_idx_1 ON `change` (changed_id, system, undone, parent_id, type, date_created);
CREATE INDEX change_idx_2 ON `change` (parent_id, system, other_side, changed_id);
CREATE INDEX change_idx_3 ON `change` (author_id, other_side, system, date_created);

The expected load is in the tens of users so the CPU and memory can be rather small (db.t2.micro or db.t2.micro equivalent). An average database can consume about 2 GB of disk space.

Daily backups are expected to enabled for the database.

Search

The Model Catalogue uses Elasticsearch server as a search engine. Currently version 2.3.5 is in use and the Catalogue should be compatible with any 2.3.x release but there is no guarantee for later releases e.g. 2.4.x. The version of the Elastic Search server should correspond to the drivers used by the Catalogue application. The best way to achieve this is to use the metadata/registry-elasticsearch Docker image which is guaranteed to be synchronized with the main application image metadata/registry and expected Elasticsearch settings.

The search indices can be rebuilt from the database, but to retain indices between container restarts and upgrades, docker container should be run with -v [backup-path]:/usr/share/elasticsearch/data

If the main Catalogue application and the Elastic Search server are not running on the same Docker engine,, then the port 9300 needs to be published with -p 9300:9300.

To achieve the best performance 4 GB of system memory is recommended. You can achieve this by running the container with the parameters --memory="4g" -e ES_HEAP_SIZE=2g.

See How to Make a Dockerfile for Elasticsearch for further information.

Here’s the summary of recommended docker run parameters (the port setting is omitted as running within the same engine is recommended):

docker run -d --restart="always" --name=mc-es -v "$PWD/esdata":/usr/share/elasticsearch/data --memory="4g" -e ES_HEAP_SIZE=2g metadata/registry-elasticsearch:2

Application

The Model Catalogue is a Java web application running on Tomcat 8 and JDK 8. It is distributed as Docker image metadata/registry which contains the application deployed as ROOT application and the Model Catalogue configuration file enabled to get important configuration from the environmental variables. You can find reference for all the environmental variables at the end of this section.

The average memory consumption of the application is under 2 GB, but some of the processes such as search reindexing or producing some of the reports may require more memory to run. Because of this the recommended memory setting for production environment is 8 GB.

CPU consumption is usually low.

The application can store Assets (blob files) in database (not recommended), file system (needs additional backup settings) or to Amazon S3 (preferred).

Mail server configuration is needed to enable user self-registration and sending forgotten password.

Users can log in with their Google accounts if Google API key and secret is set.

As the number of environmental variables is high the best way to supply it to the docker command is with an environment file: --env-file mc.env. The file should contain variable declarations in the format ENV_VAR=value on each line.

For most secure communication with the Elasticsearch server run the Elasticsearch container within the same docker engine and link it with --link mc-es:mc-es.

docker run -d --restart="always" --name=mc --env-file mc.env metadata/registry:2

Table 1. Available Environmental Variables

Group	Name	Description	Default Value
Application	CATALINA_OPTS	Additional options for Tomcat. Use it to tune memory settings e.g. -Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -Xms2g -Xmx8g -XX:NewSize=1g -XX:MaxNewSize=1g -XX:PermSize=1g -XX:MaxPermSize=1g -XX:+DisableExplicitGC	None
Application	METADATA_HOST	The URL where the application is available for the users	Value of VIRTUAL_HOST variable for easy integration with nginx reverse proxy image, value of DOCKER_MACHINE_IP variable or the result of calling http://checkip.amazonaws.com service (in this order of precedence)
Application	METADATA_SCHEME	The preferred scheme of the application	'http'
Application	MC_SECURED_REVERSE_PROXY	If set to any value then the Spring Security accepts http as secure channel as well	none
Application	MC_MAX_ACTIVE_USERS	Sets the maximum number of active (enabled) users to given number	None
Application	MC_PRELOAD	JSON containing list of objects, each having name and url	None
Application	MC_SUPERVISOR_EMAIL	Sets the email for the superuser	Value of MC_MAIL_FROM
Application	MC_ADMIN_EMAIL	Sets the email for the admin user	None
Database	METADATA_PASSWORD	The password for the Model Catalogue database	Value of MC_MYSQL_ENV_MYSQL_PASSWORD variable (set automatically when running from docker-compose) or value of RDS_PASSWORD variable (set automatically when running on Amazon ElasticBeanstalk) or metadata
Database	METADATA_USERNAME	The username for the Model Catalogue database	Value of MC_MYSQL_ENV_MYSQL_USER variable (set automatically when running with docker-compose) or value of RDS_USERNAME variable (set automatically when running on Amazon ElasticBeanstalk) or metadata
Database	METADATA_JDBC_URL	The JDBC connection string	Set automatically if running with docker-compose or Amazon ElasticBeanstalk or uses local H2 database if not set as "${System.properties['catalina.base']}/db"
Search	MC_ES_HOST	Hostname where Elasticsearch server is running	Set automatically if running with docker-compose or uses local embedded instance if not set at "${System.properties['catalina.base']}/es"
Search	MC_ES_PREFIX	Prefix of the indicies for current application (useful for shared Elasticsearch servers)	None
Mail	MC_MAIL_FROM	Default mail sender for the application	None
Mail	MC_MAIL_HOST	Hostname of the mail server	None
Mail	MC_MAIL_PORT	Port of the mail server	None
Mail	MC_MAIL_USERNAME	Username for themail server	None
Mail	MC_MAIL_PASSWORD	Password for the mail server	None
Mail	MC_MAIL_PROPS	JSON containing additional mail properties (see Grails Mail Plugin)	None
Assets	MC_S3_BUCKET	Name of the Amazon S3 bucket to store the assets	None
Assets	MC_S3_KEY	AWS Access Key for S3 access	Value of variable AWS_ACCESS_KEY_ID
Assets	MC_S3_SECRET	AWS Secret Key for S3 access	Value of variable AWS_SECRET_KEY
Assets	MC_S3_REGION	AWS Region to store the assets	None
Assets	MC_USE_LOCAL_STORAGE	Stores assets locally to the file system at "${System.properties['catalina.base']}/storage" instead of the database. Use this option only if you can’t store assets to Amazon S3!	By default the assets are stored inside the database (not recommended)
Appearance	MC_NAME	Name of the application displayed in the menu toolbar	None
Appearance	MC_WELCOME_JUMBO	HTML text to appear to users not logged in on the home page (inside big gray box)	None
Appearance	MC_WELCOME_INFO	HTML text to appear to users not logged in on the home page (bellow big gray box)	None
Authentication	MC_GOOGLE_KEY	Google API client key to enable users to log in with Google credentials	None
Authentication	MC_GOOGLE_SECRET	Google API client secret to enable users to log in with Google credentials	None
Authentication	MC_ALLOW_SIGNUP	Enables user self-registration	None