diff --git a/README.md b/README.md index 6939b62..7793198 100644 --- a/README.md +++ b/README.md @@ -2,47 +2,95 @@ This repo provides the Ansible playbook for a [Galaxy](https://galaxyproject.org/) server component on SURF ResearchCloud (SRC). -### Roles used +Also see the [official Galaxy training manual for SRC](https://training.galaxyproject.org/training-material/topics/admin/tutorials/surf-research-cloud-galaxy/tutorial.html). -* The official [Galaxy Ansible role](https://github.com/galaxyproject/ansible-galaxy) -* The official [Galaxy postgres](https://github.com/galaxyproject/ansible-postgresql) and [postgres_objects](https://github.com/galaxyproject/ansible-postgresql-objects) roles for database management -* Utility roles from the [UU SRC collection](https://github.com/UtrechtUniversity/researchcloud-items). -* Assumes Nginx is already installed on the workspace (via the SURF Nginx component) +### Prerequisites + +* Assumes Nginx is already installed on the workspace via the SURF [Nginx component](https://gitlab.com/rsc-surf-nl/plugins/plugin-nginx). ## Overview ### Services -* Postgres is installed on the same host as Galaxy +* Postgres is installed on the same host as Galaxy. * Galaxy is started the first time via `/usr/local/bin/galaxyctl start`. After that, it can be managed using `systemctl`, e.g. with `systemctl status galaxy.target`. + * Galaxy runs on `localhost:8080`. * An Nginx reverse proxy is started serving the Galaxy application on `https://`, providing SRAM authentication (see below). - -**Note: when starting the workspace, the webserver comes up before the Galaxy application has fully started. This may result in an '502 Bad Gateway' error displayed in your browser.** If this happens, just try again a minute or two later! (Todo is to give a more useful error message to the user). + * Nginx proxies port 443 to localhost:8080 +* If `src_galaxy_interactive_tools` is enabled, the [gx-it-proxy](https://github.com/galaxyproject/gx-it-proxy) application is started on port `8001`. +* If `src_galaxy_enable_tus`, [`tusd`](https://training.galaxyproject.org/training-material/topics/admin/tutorials/tus/tutorial.html) is started on port `8002`. ### Authentication -Galaxy is configured to let the webserver (Nginx) handle authentication. The webserver uses openid to authenticate against SRAM. +Galaxy is configured to let the webserver (Nginx) handle authentication. The webserver uses OpenID connect to authenticate against SRAM. Any members of the workspace's Collaborative Organisation (CO) will be able to authenticate using the authentication mechanism of their institution (Single Sign-On). Galaxy is configured such that members of the CO that are in the SRAM workspace admin group (`src_co_admin`) will be Galaxy administrators. Other users are normal users. -### ResearchCloud parameters +### Logging in via SSH + +The Galaxy VM will be accessible via SSH for users in the Collaborative Organization. However, only users in the CO admin group +(configured by the `src_galaxy_co_admin_group` parameter, default: `src_co_admin`) will be able to `sudo`. + +The Galaxy application listening on localhost will expect a secret key in the request header that Nginx is configured to pass on when reverse proxying; as non-admin users +on the Galaxy machine won't have access to this secret key, they cannot directly query Galaxy on localhost. + +## ResearchCloud parameters The component takes the following parameters: * `src_galaxy_version`: String. Set to e.g. `23.2` (default) to control the version of Galaxy that will be installed. * `src_galaxy_api_exposed`: Boolean. if `true` (default), the `/api` route does not require authentication via Single Sign-On. -* `src_ibridges`: Boolean (default: `true`). Whether to enable support for the [iBridges](https://github.com/UtrechtUniversity/galaxy-tools-ibridges) tool for connecting to Yoda and iRODS instances. Implies `src_galaxy_bootstrap`, and adds iBridges to the list of tools to be installed automatically. +* `src_ibridges`: Boolean (default: `true`). Whether to enable support for the [iBridges](https://github.com/UtrechtUniversity/galaxy-tools-ibridges) tool for connecting to Yoda and iRODS instances. Implies `src_galaxy_bootstrap`, and adds iBridges to the list of tools to be installed automatically. To use the interactive iBridges tool, don't forget to enable interactive tools as well. * `src_galaxy_jobs_default`: String. What runner to use for jobs by default. Valid values: `singularity`, `docker`, `local`. * `src_galaxy_interactive_tools`: if `true` (default), support for [interactive tools](https://docs.galaxyproject.org/en/master/admin/special_topics/interactivetools.html) is enabled. * `src_galaxy_co_admin_group`: String group corresponding to an SRAM group. Members of this SRAM group will be made Galaxy admin users. -* `src_galaxy_bootstrap`: if `true` (default), will attempt to install workflows, dataproviders and tools as configured by the following options: +* `src_galaxy_bootstrap`: Boolean. Also see [below](#bootstrapping). If `true` (default), will attempt to install workflows, dataproviders and tools as configured by the following options: * `src_galaxy_tool_files`: String of comma-separated paths to YAML files (in this repo) containing tool specifications. * `src_galaxy_workflow_files`: String of comma-separated paths to `.ga` files (in this repo) containing workflow specifications. - * `src_galaxy_custom_repo`: **Interactive parameter**. String URL to a git repo containing workflow files (in the `workflows` dir of the root of the repo) and tool specification files (in the `tools` dir). Example repo: - * `src_galaxy_custom_repo_branch`: **Interactive parameter**. String branch of the custom repo to be used. -* `src_galaxy_storage_path`: **Interactive parameter**. Default: `/srv/galaxy/datadir`. Path where Galaxy's "mutable data directory" will be located, including the following files: + * `src_galaxy_custom_repo`: String URL to a git repo containing workflow files (in the `workflows` dir of the root of the repo) and tool specification files (in the `tools` dir). Example repo: + * `src_galaxy_custom_repo_branch`: String branch of the custom repo to be used. +* `src_galaxy_storage_path`: String filepath. Default: `/srv/galaxy/datadir`. Path where Galaxy's "mutable data directory" will be located. If you attach additional networked storage to the workspace, you can set `src_galaxy_storage_path` to a path on that storage volume. If your storage is e.g. called "galaxy storage", set the parameter to: `/data/galaxy_storage/datadir`. In theory, this should allow you to re-use datasets, tools, etc. from previous Galaxy workspaces. +* `src_galaxy_enable_tus`: Boolean. If true, enable support for performant and resumable uploads using [tusd](https://training.galaxyproject.org/training-material/topics/admin/tutorials/tus/tutorial.html). + +## Bootstrapping + +The `src_galaxy_bootstrap` parameter determines whether the Galaxy instance should be bootstrapped: that is, whether various tools, workflows, and datamanagers should be installed. +These can be installed from various sources: + +* a git repo containing `.yml` files descibing which tools to install, and `.ga` workflows files + * example repo here + * put tool files in the `tools` subdirectory (as defined by the `_galaxy_custom_repo_tool_location` internal variable). + * put workflow files in the `workflows` subdirectory (as defined by the `_galaxy_custom_repo_workflow_location` internal variable). + * use `src_galaxy_custom_repo` and `src_galaxy_custom_repo_branch` to define your repository. +* any number of `.yml` tool files found in this repository + * use `src_galaxy_tool_files` to provide these. For instance, you may set this parameter to `tools/ibridges.yml,tools/foo.yml` to install the tools defined in those locations. + * see [here](tools/ibridges.yml) for an example +* any number of `.ga` workflow files found in this repository + * use `src_galaxy_workflow_files` to provide these. For instance, you may set this parameter to `tools/bar.ga` to install that workflow. + +By customizing the `src_galaxy_custom_repo`, `src_galaxy_tool_files` and `src_galaxy_workflow_files` parameters, you can thus create different 'flavours' for your Galaxy Catalog Item +that each contain different tools and workflows out of the box. + +__Note__: workflow files installed using the bootstrapping procedure will be installed as `public` workflows available to all users on the Galaxy instance! +You can find these under `Data > Workflows` in Galaxy's main menu. + +### Bootstrapping: how does it work? + +If boostrapping is enabled, this Ansible playbook will: + +1. Configure Galaxy to use a bootstrapping API key in `galaxy.conf`. +1. Restart Galaxy. + +...and then invoke the [ansible-galaxy-tools role](https://github.com/UtrechtUniversity/ansible-galaxy-tools/) to do the following: + +1. Use the bootstrapping API key to import tools. +1. Use the bootstrapping API key to create an admin user. + * Use an API key belonging to the admin user to import workflows. + +If needed Galaxy, is restarted. Finally, the playbook will __always__ remove the bootstrapping API key. -If you attach additional networked storage to the workspace, you can set `src_galaxy_storage_path` to a path on that storage volume. If your storage is e.g. called "galaxy storage", set the parameter to: `/data/galaxy_storage/datadir`. In theory, this should allow you to re-use datasets, tools, etc. from previous Galaxy workspaces. +## Maintenance +You can stop/restart Galaxy using systemctl, e.g.: ` systemctl restart galaxy.target`. Use e.g. ` galaxyctl follow` to follow the logs of all Galaxy services. diff --git a/galaxysrv.yml b/galaxysrv.yml index 11912ba..06d5ac8 100644 --- a/galaxysrv.yml +++ b/galaxysrv.yml @@ -39,6 +39,8 @@ - role: galaxyproject.galaxy vars: galaxy_config: "{{ _galaxy_config }}" + - role: galaxyproject.tusd + when: _galaxy_enable_tus tasks: @@ -59,7 +61,7 @@ dest: "{{ galaxy_config_dir }}/tool_conf.xml" owner: "{{ galaxy_privsep_user }}" group: "{{ galaxy_user }}" - mode: "644" + mode: "0644" - name: Enable interactive tools ansible.builtin.include_tasks: tasks/interactive_tools.yml @@ -83,10 +85,10 @@ - name: Populate service facts ansible.builtin.service_facts: - # This is to ensure that the co admin group can use passwordless sudo on the machine, + # This is to ensure that the Collaborative Organisation admin group can use passwordless sudo on the machine, # allowing us to disable the ResearchCloud co_passwordless_sudo parameter, # which grants passwordless sudo to *all* CO users on the machine. - - name: Add admin group to sudoers + - name: Add CO admin group to sudoers ansible.builtin.copy: dest: /etc/sudoers.d/co_admins owner: "root" diff --git a/molecule/default/converge.yml b/molecule/default/converge.yml index 5b2faeb..51b2e81 100644 --- a/molecule/default/converge.yml +++ b/molecule/default/converge.yml @@ -9,3 +9,4 @@ src_galaxy_tool_files: "{{ playbook_dir ~ '/molecule/_testfiles/tool_list.yaml.sample' }}" src_galaxy_workflow_files: "{{ playbook_dir ~ '/molecule/_testfiles/sample-workflow.ga' }}" src_ibridges: 'true' + workspace_fqdn: localhost diff --git a/molecule/default/molecule.yml b/molecule/default/molecule.yml index 604d172..8ceceb1 100644 --- a/molecule/default/molecule.yml +++ b/molecule/default/molecule.yml @@ -16,7 +16,7 @@ platforms: volumes: - /sys/fs/cgroup:/sys/fs/cgroup:rw published_ports: - - 8080/tcp + - 80/tcp provisioner: name: ansible config_options: @@ -33,7 +33,6 @@ scenario: - create - prepare - converge - - side_effect # - idempotence # disabling idempotence because the remote_user_secret option is not idempotent - verify - cleanup diff --git a/molecule/default/side_effect.yml b/molecule/default/side_effect.yml deleted file mode 100644 index 418253d..0000000 --- a/molecule/default/side_effect.yml +++ /dev/null @@ -1,12 +0,0 @@ -- name: Side Effect - hosts: all - gather_facts: false - - tasks: - - name: Make sure Galaxy is fully restarted before running idempotence - ansible.builtin.uri: - url: "http://localhost:8080/api/version" - register: galaxy_service_live - until: "galaxy_service_live.status == 200" - retries: 30 - delay: 10 # check every 10 seconds for a maximum of 5 minutes diff --git a/molecule/default/verify.yml b/molecule/default/verify.yml new file mode 100644 index 0000000..a572a15 --- /dev/null +++ b/molecule/default/verify.yml @@ -0,0 +1,23 @@ +- name: Verify + hosts: all + gather_facts: false + + tasks: + - name: Make sure Galaxy API is up + ansible.builtin.uri: + url: "http://localhost:8080/api/version" + register: galaxy_service_live + until: "galaxy_service_live.status == 200" + retries: 10 + delay: 10 # check every 10 seconds for a maximum of 5 minutes + + - name: Test reverse proxy + ansible.builtin.command: curl --fail-with-body http://localhost:80/ + changed_when: false + failed_when: false + register: galaxy_service_live + + - name: Make sure Nginx is reverse proxying to Galaxy + ansible.builtin.assert: + that: + - "'Access to Galaxy is denied' in galaxy_service_live.stdout" # this sentence is displayed by Galaxy if the GX_SECRET header is not passed along diff --git a/requirements.yml b/requirements.yml index 3dee623..3ab0219 100644 --- a/requirements.yml +++ b/requirements.yml @@ -3,7 +3,7 @@ roles: - src: https://github.com/galaxyproject/ansible-galaxy.git name: galaxyproject.galaxy version: 08974b18e2aad1e5569854e5d6c8b5c966bf55e4 - - src: https://github.com/dometto/ansible-galaxy-tools.git + - src: https://github.com/utrechtuniversity/ansible-galaxy-tools.git name: galaxyproject.galaxy-tools - src: galaxyproject.postgresql version: 1.1.2 @@ -11,10 +11,13 @@ roles: version: 1.2.0 - src: geerlingguy.docker version: 6.1.0 + - name: galaxyproject.tusd + src: https://github.com/galaxyproject/ansible-role-tusd.git + version: e009b498a7989d8002c6a5d104176295d63e9fae collections: - name: community.general - name: community.postgresql - - name: https://github.com/UtrechtUniversity/researchcloud-items.git + - name: https://github.com/UtrechtUniversity/researchcloud-items.git # uusrc.general type: git - version: 02dab70feb046030e46e55886c2afa8ae7982c08 + version: 801c6d2cb21677ef3926c4172cd6c79e48004b62 diff --git a/tasks/bootstrap.yml b/tasks/bootstrap.yml index 7189f17..482dcc2 100644 --- a/tasks/bootstrap.yml +++ b/tasks/bootstrap.yml @@ -2,6 +2,8 @@ - name: Wait for galaxy server to come up ansible.builtin.uri: url: http://{{ _galaxy_local_address }}/api/version + headers: + GX_SECRET: "{{ _galaxy_remote_user_secret }}" register: galaxy_service_live until: galaxy_service_live.status == 200 retries: 30 diff --git a/templates/galaxy/config/dependency_resolvers_conf.xml.j2 b/templates/galaxy/config/dependency_resolvers_conf.xml.j2 new file mode 100644 index 0000000..18a55ac --- /dev/null +++ b/templates/galaxy/config/dependency_resolvers_conf.xml.j2 @@ -0,0 +1,2 @@ + + \ No newline at end of file diff --git a/vars/galaxy_vars.yml b/vars/galaxy_vars.yml index 7f88952..877a644 100644 --- a/vars/galaxy_vars.yml +++ b/vars/galaxy_vars.yml @@ -9,6 +9,8 @@ galaxy_venv_dir: "{{ _galaxy_root }}/venv" galaxy_config_dir: "{{ _galaxy_root }}/config" galaxy_mutable_data_dir: "{{ src_galaxy_storage_path | default(_galaxy_root + '/datadir', true) }}" # component variable src_galaxy_storage_path galaxy_commit_id: release_{{ src_galaxy_version | default('24.1', true) }} # component variable src_galaxy_version +galaxy_tusd_port: 8002 +galaxy_tus_upload_store: "{{ galaxy_mutable_data_dir }}/tus_data" galaxy_separate_privileges: true galaxy_create_user: true galaxy_manage_paths: true @@ -24,6 +26,8 @@ galaxy_config_templates: dest: "{{ galaxy_config_dir }}/user_preferences_extra_conf.yml" - src: templates/galaxy/config/job_conf.yml.j2 dest: "{{ galaxy_config_dir }}/job_conf.yml" + - src: templates/galaxy/config/dependency_resolvers_conf.xml.j2 + dest: "{{ galaxy_config_dir }}/dependency_resolvers_conf.xml" postgresql_objects_users: - name: galaxy password: @@ -46,8 +50,10 @@ galaxy_configuration: interactivetools_enable: "{{ _galaxy_use_interactive_tools }}" interactivetools_map: "{{ _galaxy_use_interactive_tools | ternary(gie_proxy_sessions_path, omit) }}" galaxy_infrastructure_url: "{{ _galaxy_local_address }}" + galaxy_tus_upload_store: "{{ galaxy_tus_upload_store }}" outputs_to_working_directory: "{{ _galaxy_use_interactive_tools }}" bootstrap_admin_api_key: "{{ _galaxy_bootstrap_api_key | default(omit) }}" + nginx_x_accel_redirect_base: '/_x_accel_redirect' gravity: galaxy_root: "{{ galaxy_server_dir }}" galaxy_user: "{{ galaxy_user }}" @@ -76,3 +82,8 @@ galaxy_configuration: gx_it_proxy: enable: "{{ _galaxy_use_interactive_tools }}" port: "{{ gie_proxy_port }}" + tusd: + enable: "{{ _galaxy_enable_tus }}" + tusd_path: /usr/local/sbin/tusd + upload_dir: "{{ galaxy_tus_upload_store }}" + port: "{{ galaxy_tusd_port }}" diff --git a/vars/interactive_tools_vars.yml b/vars/interactive_tools_vars.yml index d06e8eb..d84cf29 100644 --- a/vars/interactive_tools_vars.yml +++ b/vars/interactive_tools_vars.yml @@ -2,4 +2,4 @@ gie_proxy_dir: "{{ _galaxy_root }}/gie-proxy/proxy" gie_proxy_sessions_path: "{{ galaxy_mutable_data_dir }}/interactivetools_map.sqlite" gie_proxy_path_prefix: /interactivetool/ep -gie_proxy_port: 4002 +gie_proxy_port: 8001 diff --git a/vars/nginx_vars.yml b/vars/nginx_vars.yml index 51fa395..4bd6190 100644 --- a/vars/nginx_vars.yml +++ b/vars/nginx_vars.yml @@ -7,6 +7,13 @@ galaxy_nginx_vhost_config: auth: "{{ _molecule_active | ternary(omit, 'sram') }}" proxy_set_header: GX_SECRET: "{{ _galaxy_remote_user_secret | default(omit) }}" + gzip: 'on' + gzip_http_version: 1.1 + gzip_vary: 'on' + gzip_comp_level: 6 + gzip_proxied: any + gzip_types: text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript + gzip_buffers: 16 8k - name: api location: /api/ proxy_pass: http://{{ _galaxy_local_address }}/api/ @@ -33,3 +40,22 @@ galaxy_nginx_vhost_config: location: = /galaxy502.html root: "{{ _galaxy_error_502 | dirname }}" internal: "" + - name: x_accel_redirect + location: /_x_accel_redirect/ + internal: "" + alias: / + add_headers: + Access-Control-Allow-Origin: $upstream_http_access_control_allow_origin + Access-Control-Allow-Methods: $upstream_http_access_control_allow_methods + - name: tus + location: '/api/upload/resumable_upload' + proxy_request_buffering: 'off' + proxy_buffering: 'off' + proxy_http_version: '1.1' + proxy_pass: http://localhost:{{ galaxy_tusd_port }}/files + proxy_set_header: + X-Forwarded-Host: '$host' + X-Forwarded-Proto: '$scheme' + Upgrade: '$http_upgrade' + Connection: 'Upgrade' + client_max_body_size: '0' diff --git a/vars/src_galaxy_vars.yml b/vars/src_galaxy_vars.yml index 8bd6d0a..9083fa0 100644 --- a/vars/src_galaxy_vars.yml +++ b/vars/src_galaxy_vars.yml @@ -15,6 +15,8 @@ _galaxy_use_interactive_tools: "{{ src_galaxy_interactive_tools | default(true, _galaxy_server_fqdn: "{{ workspace_fqdn | default('localhost', true) }}" _galaxy_jobs_default: "{{ src_galaxy_jobs_default | default('local', true) }}" _galaxy_enable_docker: "{{ _galaxy_use_interactive_tools or (_galaxy_jobs_default == 'docker') }}" +_galaxy_enable_tus: "{{ src_galaxy_enable_tus | default(true, true) | bool }}" +_galaxy_external_url: "https://{{ workspace_fqdn }}" # Build the final lists of tool and workflow installation files