Test and production environments

Get started

1. Install Docker;
2. In your command line launch the following commands:

git clone https://github.com/Wikidata/soweego.git
cd soweego

Test environment

It's useful when you are developing or testing some features. In this environment, you don't need to be afraid of breaking stuff.

It provides you with a MariaDB instance and a BASH shell ready to run soweego CLI commands. You are free to change soweego code while the shell is running, the code is synced.

What do you need to run it?

• Docker
• Docker Compose

How do you run it?

1. In your terminal, move to the project root;
2. Launch ./scripts/docker/launch_test.sh;
3. You are now in a Docker container BASH shell with a fully working soweego instance;
4. Run cd soweego;
5. You are set. To check if it's working, try python -m soweego.

launch_test.sh options

Option	Expected Value	Default Value	Description
-s	directory path	/tmp/soweego_shared	Tells docker which folder in your machine will be shared with soweego container.

How do you connect with the local database instance?

The test environment comes with a running MariaDB instance. To query it from your terminal:

1. docker exec -it soweego_db_1 /bin/bash;
2. mysql -uroot -hlocalhost -pdba soweego.

Production environment

It's useful when you need to run soweego against the Wikimedia database. It is also helpful to run the system against a custom database. Editing the credentials is all you need to do to chose the database. Note: you need access to the Wikimedia infrastructure to run soweego on it.

This environment provides you with a BASH shell ready to run soweego CLI commands. You are free to change soweego code while the shell is running, the code is synced.

What do you need to run it?

• Docker
• Database credentials file, like ${PROJECT_ROOT}/soweego/importer/resources/db_credentials.json.

How do you run it?

1. In your terminal, move to the project root;
2. Launch ./scripts/docker/launch_prod.sh;
3. You are now in a Docker container BASH shell with a fully working soweego instance;
4. Run cd soweego;
5. You are set. To check if it's working, try python -m soweego.

launch_prod.sh options

Option	Expected Value	Default Value	Description
-s	directory path	/tmp/soweego_shared	Tells docker which folder in your machine will be shared with soweego container.
-c	file path	${PROJECT_ROOT}/soweego/importer/resources/db_credentials.json	Sets which file in your machine soweego will read for database credentials.

Pipeline

It's useful when you need to run all the soweego steps. It does not require you to have the project in your machine: it is an already built docker image. Editing the credentials is all you need to do to chose the database.

This environment is a docker container that executes python -m soweego pipeline command. Obviously, it can be run with custom options. Note: the code is put into the image during the build process.

What do you need to run it?

• Docker
• Database credentials file, like ${PROJECT_ROOT}/soweego/importer/resources/db_credentials.json.

How do you run it?

1. In your terminal, move to the project root;
2. Launch ./scripts/docker/launch_pipeline.sh;
3. The script will ask you some parameters.
4. The pipeline is now running!

launch_pipeline.sh options

Option	Expected Value	Default Value	Description
-s	directory path	/tmp/soweego_shared	Tells docker which folder in your machine will be shared with soweego container.
target (argument, not an option)	one of the available targets name (atm. musicbrainz, discogs. imdb)	None	Tells soweego which database will go through the whole process.
--validator / --no-validator	No value needed. The option you choose among the two is the value itself.	--no-validator	Tells soweego to run or not the validator step.
--importer / --no-importer	No value needed. The option you choose among the two is the value itself.	--importer	Tells soweego to run or not the importer step.
--linker / --no-linker	No value needed. The option you choose among the two is the value itself.	--linker	Tells soweego to run or not the linker step.
--upload / --no-upload	No value needed. The option you choose among the two is the value itself.	--upload	Tells soweego to upload or not the computed links among wikidata and the target to wikidata.