From 6c73c42dabbfce76e0fe854851cabd665f16be5c Mon Sep 17 00:00:00 2001 From: Zhanna Klimanova Date: Mon, 3 Jun 2024 23:33:47 -0400 Subject: [PATCH] Adding clarifications and further details on installation instructions for local development --- README.md | 96 +++++++++++++++++++++++++++++++++---------------------- 1 file changed, 58 insertions(+), 38 deletions(-) diff --git a/README.md b/README.md index 9531e52..8c81966 100644 --- a/README.md +++ b/README.md @@ -5,78 +5,98 @@ The ambitious initiative will greatly assist in mapping instrument names and fac ![UMIL-1-1](https://github.com/DDMAL/VIM/assets/61984039/cf808948-11be-459b-9060-55220dbbade6) +## Table of Contents +- [Installation for Local Development](#installation-for-local-development) + - [Debugging](#debugging) + - [Tools for Python Development](#tools-for-python-development) +- [Installation for Deployment](#installation-for-deployment) +- [Managing Database Migrations](#managing-database-migrations) +- [Loading Data](#loading-data) + ## Installation for Local Development NOTE: These instructions are for local development only. Refer to the "Installation for Deployment" section for installation on a remote server. -UMIL requires Docker Engine with Compose V2. UMIL's Docker Compose configuration is written according to the Compose Specification. - -After cloning this repository, set up a local `.env` file. Copy or rename the `.env.sample` file to `.env` and update it to include uncommented environment variables for database credentials `POSTGRES_USER` and `POSTGRES_PASSWORD` and the `DJANGO_SECRET_KEY`. Verify the values of the `DEVELOPMENT`, `HOST_NAME`, and `PORT` variables. For local development ONLY, these should be set to "true", "localhost", and "8000" respectively. +### Requirements +- UMIL requires Docker Engine with Compose V2. Verify your version with `docker compose --version` and look for an output such as `Docker Compose version v2.19.1`. If you do not have the correct version, refer to the [Docker Compose Migration Documentation](https://docs.docker.com/compose/migrate/). -```console -> docker compose build -> docker compose up +### Initial Set-up Instructions +After cloning this repository, you will need a local `.env` file at the root of the directory. Copy the contents or rename the `.env.sample` file to `.env` and update it to include uncommented environment variables for the database credentials `POSTGRES_USER` and `POSTGRES_PASSWORD`, as well as the `DJANGO_SECRET_KEY`. The database credentials can be set to anything while the `DJANGO_SECRET_KEY` will need to be obtained from one of the developers working on the UMIL project. Next you will need to verify the values of the `DEVELOPMENT`, `HOST_NAME`, and `PORT` variables. For local development ONLY, these should be set to "true", "localhost", and "8000" respectively. \ +In one terminal run the following commands to build the docker images and run the specified services. +```sh +docker compose build +docker compose up +``` +If you are not running docker containers in detatched mode (e.g `docker compose up -d`) open a second terminal and go into the running container shell. Run migrations commands to create/populate tables. The following commands should be executed to complete the aforementioned steps. +```sh +docker compose exec -it app bash +python manage.py makemigrations +python manage.py migrate +python manage.py import_languages +python manage.py import_instruments ``` -The django development server should now be available at `localhost:8000`. - -### Debugging - -When installed with `DEVELOPMENT=true` in the `.env` file, some additional modules are included in the application container that provide debugging tools: `django-extensions` and `django-debug-toolbar`. The development installation runs the `runserver_plus` (provided by `django-extensions`) command automatically, which provides an in-browser debug console when errors are raised in the application. `django-extensions` also provides additional tools potentitally useful for development; see the [module documentation](https://django-extensions.readthedocs.io/en/latest/command_extensions.html) for more. The debugging toolbar provided by `django-debug-toolbar` is shown when viewing the development server in the browser. - -### Tools for Python Development +The Django development server should now be available at `localhost:8000`. Ensure that the database schema is properly set up and the application can display data as expected by navigating to `localhost:8000/instruments/`. -UMIL uses `poetry` to manage python dependencies and a `poetry.lock` file to ensure reproducible builds in development and production. Dependencies can be found in the `pyproject.toml` file and are divided into three groups: +NOTE: that it is recommended to bring the containers down with `docker compose down` after you are done developing, testing, and committing your changes. After executing the previous command, the application will no longer be available `localhost:8000`. To once again view the running application, run `docker compose up -d` and ensure all migrations made by you or other developers are up-to-date with `docker compose exec -it app bash` and `python manage.py migrate`. For further clarifications on managing migrations refer to this [section](#managing-database-migrations). -1. `main` - the core dependencies required by the application -2. `debug` - dependencies that provide tools for debugging, and that are installed in the application Docker container only during local development -3. `dev` - optional dependencies (eg. for code formatting, linting, type checking) that can be useful in the development environment, but are never used in the application container +### Debugging -It is not generally necessary to use `poetry` for development, except when adding additional dependencies. +When installed with `DEVELOPMENT=true` in the `.env` file, some additional modules are included in the application container that provide debugging tools: `django-extensions` and `django-debug-toolbar`. The debugging toolbar provided by `django-debug-toolbar` is shown when viewing the development server in the browser. The development installation runs the `runserver_plus` (provided by `django-extensions`) command automatically, which provides an in-browser debug console when errors are raised in the application. `django-extensions` also provides additional tools potentitally useful for development. Further information can be found at [module documentation](https://django-extensions.readthedocs.io/en/latest/command_extensions.html). ## Installation for Deployment -UMIL requires Docker Engine with Compose V2. Ensure that the remote server has these installed. +UMIL requires Docker Engine with Compose V2. Ensure that the remote server has these installed. -SSH into the server. After cloning the repository, set up a local `.env` file. Copy or rename the `.env.sample` file to `.env` and update it to include uncommented environment variables for database credentials `POSTGRES_USER` and `POSTGRES_PASSWORD`. Ensure that `POSTGRES_PASSWORD` is secure. Additionally, set `DJANGO_SECRET_KEY` with a secure secret key for Django. +SSH into the server. After cloning the repository, set up a local `.env` file. Copy or rename the `.env.sample` file to `.env` and update it to include uncommented environment variables for database credentials `POSTGRES_USER` and `POSTGRES_PASSWORD`. Ensure that `POSTGRES_PASSWORD` is secure. Additionally, set `DJANGO_SECRET_KEY` with a secure secret key for Django. Ensure that the `DEVELOPMENT` variable is set to "false", and that `HOST_NAME` is set to the host name where the UMIL instance will be served (for example, "vim.linkedmusic.ca" or "vim.staging.linkedmusic.ca"). Set the `PORT` variable to "80". - -Then, run +Run the following commands: ```bash -> docker compose build -> docker compose up -d +docker compose build +docker compose up -d ``` ## Managing Database Migrations Django automates changes to the database schema with migrations. -If your development changes any application models, be sure to make and commit migrations along with those changes. To create migrations, enter the application container and run the `makemigrations` command: +If your development changes any application models, be sure to make and commit migrations along with those changes. To create and apply migrations open a separate terminal and run the following commands: -```console -> user@machine:~/dev-directory$ docker compose exec -it app bash -> root@container-id:/virtual-instrument-museum/vim-app$ python manage.py makemigrations +```sh +docker compose exec -it app bash +python manage.py makemigrations +python manage.py migrate ``` Commit the resulting migrations files with the model changes. If changes you make require migrations, or you merge migrations made by others into you development branch, you will need to apply those migrations to your local copy of the database: -```console -> user@machine:~/dev-directory$ docker compose exec -it app bash -> root@container-id:/virtual-instrument-museum/vim-app$ python manage.py migrate +```sh +docker compose exec -it app bash +python manage.py migrate ``` -## Loading data +## Loading Data At present, UMIL supports an initial list of 262 instruments curated from Wikidata, and their names in English and French. These instruments can be found in `web-app/django/startup_data/vim_instruments_with_images-15Sept.csv`. -Two django management commands are provided to load these initial instruments. These should be run from within the app Docker container. +Two django management commands are provided to load these initial instruments. These should be run from within the app Docker container by going into its shell. -```bash -> user@machine:~/dev-directory$ docker compose exec -it app bash -> root@container-id:/virtual-instrument-museum/vim-app$ python manage.py import_languages -> root@container-id:/virtual-instrument-museum/vim-app$ python manage.py import_instruments -``` \ No newline at end of file +```sh +docker compose exec -it app bash +python manage.py import_languages +python manage.py import_instruments +``` + +## Additional Tools for Python Development + +UMIL uses `poetry` to manage python dependencies and a `poetry.lock` file to ensure reproducible builds in development and production. Dependencies can be found in the `pyproject.toml` file and are divided into three groups: + +1. `main` - the core dependencies required by the application. +2. `debug` - dependencies that provide tools for debugging and are installed in the application Docker container only during local development. +3. `dev` - optional dependencies (eg. for code formatting, linting, type checking) that can be useful in the development environment, but are never used in the application container. + +NOTE: it is not generally necessary to use `poetry` for development, except when adding additional dependencies. \ No newline at end of file