cioos ckan v1.6.0 (#225)

* add projects facet

* change projects type yo stringCollection
remove unused solr config
correctly populate projects in the solr index
update solr8 config

* add more font weights to gui and cleanup repeating subfields display

* fix display of fluent fields in subfields

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* update license and match on legacy ids

* update

* Updated submodule contrib/docker/src/cioos-siooc-schema

* add a few fields to the solr index so we can include them in field list (fl) queries

* Updated submodule contrib/docker/src/ckanext-cioos_harvest

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* Updated submodule contrib/docker/src/ckanext-spatial

* Updated submodule contrib/docker/src/ckanext-spatial

* add ECV field

* update solr schema and entrypoint files

* change responsible org to groups

* Updated submodule contrib/docker/src/ckanext-cioos_harvest

* test resp. org. group and new harvest source features

* Updated submodule contrib/docker/src/cioos-siooc-schema

* Updated submodule contrib/docker/src/ckanext-spatial

* split french and english index fields so that a french specific text parsing can be used

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* update submodules

* merge in latest changes

* populate groups during ckan harvest

* fix bugs

* hide empty facets on group and organization pages

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* adjust indexing to handle french and english separately

* add update instructions

* add file rotation to default log file

* update dev branches of submodules

* add a build css script

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* Updated submodule contrib/docker/src/cioos-siooc-schema

* update dev github actions docker build

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* build cioos ckan off ckan/ckan base docker image (#224)

* [ckan#7119] package_update performance backport version

* Revert "package_update performance backport version"

* Fix 404 when selecting the same date in the changes view

In the /dataset/<id>/changes_multiple page, if the user selects
the same date in both select controls a 404 is returned.
The case when the second date was more recent than the first one
 is already handled (selecting the most recent version), we just
need to account for the case when both dates are the same.

* [i18n] Update 2.9 translation catalogue (pot)

* Restrict user and group/org image upload formats by default

Allow only a limited number of formats, that can be expanded with
`ckan.upload.user.mimetypes` and `ckan.upload.user.types` (same for
`group` instead of user). Added new tests to cover common formats used
by spammers.

* Disable public registration of users by default

Set `ckan.auth.create_user_via_web = false` as this is the most common
deployment scenario for CKAN, and leaving open the user registration
leads to Spam problems.

* Fix tests and lint

* Defaults were missing in backport to check_ckan_version

* [ckan#7119] package_update performance backport version

* [ckan#7119] filter first, then purge

* Fix "_" override

* Updated readme to include instructions to install solr using docker

* Removed duplicate links

* Replaced instructions from master branch

The manual installation instructions for solr is replaced with master branch instructions

* Removed tomcat installation part from readme

* [ckan#7230] Add dev containers / codespaces config for CKAN 2.9

* Make test_user_list_order_by_default locale-independent

* [ckan#7031] prepare_dataset_blueprint: support dataset type

* [ckan#7031] add changes file

* Fix typo in search-index command

Update typo in search-index command (rebuild-fast instead of rebuild_fast)

* Add option to output token value without extra messages

When running `ckan user token add` you get this output by default:

```
API Token created:
        eyJ0eXAiOiJKV1QiL...
```

This is user-friendly but is not great for integrating in user scripts
as you need to clean up the output a bit, eg:

```
export CKAN_API_TOKEN=$(ckan user token add ckan_user my_token | tail -n 1 | tr -d '\t')
```

The changes suggested add a `--quiet` / `-q` flag that make the command
just output the token directly so you can do:

```
export CKAN_API_TOKEN=$(ckan user token add ckan_user my_token -q)
```

* changelog

* Add pillow as dev requirement

* Add command to clear users (by @pdelboca)

Adds new command `ckan clean users` that will scan the system for users with
invalid images and delete both the image and the user from the database.

Invalid images are all images that doesn't contain allowed mimetypes set
in `ckan.upload.user.mimetypes` config option.

* Add entity type to package get_index

* Fix package_update documentation

* Delete ckan/plugins/toolkit.py.orig

* Fixed color contrast in dashboard

* Fix color contrast for "Markdown" link

* Fix color contrast for Danger button

* Color contrast for warning button

* Add config values to test since they are default starting 2.10

* Install version of Pillow compatible with py2

* Skip clean tests on py2

* lint

* Update templates.rst

* Update docs for jwt secrets'

* Merge pull request ckan#7310 from shashigharti/7193-fix-color-contrast-accessibility-in-dashboard-for-v2_9

Fix color contrast in dashboard

* Merge pull request ckan#7308 from shashigharti/7199-fix-color-contrast-accessibility-in-useredit-for-v2_9

Fix color contrast accessibility in user edit page

* [ckan#7110] Enable DateTime to be returned through Actions by @EricSoroos

* [ckan#7351] Document how to add models to SQLAlchemy metadata by @smotornyuk

* Prefix literal

* Update install-from-source.rst

* Do not auto-register prefixed resource blueprint;

* Added changelog file;

* Rename 7374.fix to 7374.feature

* Load the `resource` blueprint after all other package blueprints;

* Update changelog for 2.9.8

* Rebuild frontend

* Update .tx/config file format

To be used with the new version of the Transifex client:

https://github.com/transifex/cli

Obtained running the ``tx migrate`` command (and manually adding the
existing comments)

* [i18n] Pull po files from Transifex before 2.9.8

* [i18n] Compile mo files

* Update version number for 2.9.8

* Update version number for 2.9.9b

* [doc] Fix versions in banners and package names

The old sorting was string based, so 2.10 broke things. Also the
packages were showing the latest stable version, not the current one

* [doc] Fix versions in banners and package names

The old sorting was string based, so 2.10 broke things. Also the
packages were showing the latest stable version, not the current one

* [doc] We only support 2 versions

* [doc] We only support 2 versions

* Fix version name function

* Remove outdated documentation for upgrading postgres

* Add note about target branches in contributing docs

* [ckan#7508] Escape names in emails

Fix ckan#7508 by quoting names in To/From headers

* Fix test

* Use `email.utils.formataddr` from stdlib to format name/address pairs

* Add changelog

* Tighten checks around incoming resource ids

* Remove Python 2 tests, use Debian buster image

* Update version for 2.9.9

* Fix openjdk package version

* Use py 3.9

* Use dockerized solr in tests

* Don't use Resource factories when creating new Datasets

The Resource factory creates an actual Resource model linked to an
automatically created dataset. If trying to create a Dataset factory
passing a resource one, it will fail as the id already exists

* Use proper solr url

* pep8

* Fix another resource factory

* [ckan#7602] Mention extra requirement in migration notes for 2.10.1

* Update solr.rst

* Add RTD conf file

* Use py3 in docs builds

* Upgrade pyyaml to avoid AttributeError: cython_sources error

* remove setuptools req on docs

* Build on py 3.9

* remove core ckan code

* fix package version

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* switch to using core ckan docker image as base

* remove unneeded lines from dockerfile

* move config settings into .env file

* add  some local backup files to git ignore

* Updated submodule contrib/docker/src/ckanext-spatial

* Updated submodule contrib/docker/src/ckanext-spatial

* Updated submodule contrib/docker/src/ckanext-spatial

* update submodules and add longer delays on health checks

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* Updated submodule contrib/docker/src/ckanext-spatial

* update to using apline ckan base image

* update environment template file

---------

Co-authored-by: Ian Ward <ian@excess.org>
Co-authored-by: Sergey <tonnydexter@gmail.com>
Co-authored-by: amercader <amercadero@gmail.com>
Co-authored-by: Brett Jones <54408245+kowh-ai@users.noreply.github.com>
Co-authored-by: Jari Voutilainen <jari.voutilainen@iki.fi>
Co-authored-by: avdata99 <andres@data99.com.ar>
Co-authored-by: shashi gharti <shashi.gharti@gmail.com>
Co-authored-by: Francesco Frassinelli <francesco.frassinelli@nina.no>
Co-authored-by: Patricio Del Boca <patriciodelboca@gmail.com>
Co-authored-by: Evert Ramos <evert.ramos@gmail.com>
Co-authored-by: Jari Voutilainen <jari-pekka.voutilainen@gofore.com>
Co-authored-by: Rukshan Ranatunge <arkruka@gmail.com>
Co-authored-by: Sergey Motornyuk <sergey.motornyuk@linkdigital.com.au>
Co-authored-by: Konstantin Sivakov <konstantin.sivakov@gmail.com>
Co-authored-by: JVickery-TBS <jesse.vickery@tbs-sct.gc.ca>
Co-authored-by: Jesse Vickery <97247789+JVickery-TBS@users.noreply.github.com>
Co-authored-by: Ben Soroos <bensoroos@derilinx.com>
Co-authored-by: Brett <brett@kowh.ai>

* Updated submodule contrib/docker/src/ckanext-spatial

* update config, maybe this will make the github build work?

* limit flask version to 2.3.3 or less 3.0.3 breaks the import of _request_ctx_stack as it is removed in v3

* fix flask debug toolbar version to fix the rest

* fix default config missing key

* Updated submodule contrib/docker/src/ckanext-spatial

* use ckan base startup script.

* remove unneeded config settings as they are set in the .env now

* Updated submodule contrib/docker/src/ckanext-spatial

* update the docs. WIP

* reformat plugins list so it is replaced on ckan start correctly

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* remove hardcoded site url from ckan build process

* Updated submodule contrib/docker/src/ckanext-spatial

* Updated submodule contrib/docker/src/ckanext-spatial

* fix related works and add lineage

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* add to update instructions

* update instructions again

* clean up instructions and config variables a little. remove secrets from production.ini so that they are read from .env file

* fix sitemap generation and bump docker-compose version

* update to production versions of submodules

---------

Co-authored-by: Ian Ward <ian@excess.org>
Co-authored-by: Sergey <tonnydexter@gmail.com>
Co-authored-by: amercader <amercadero@gmail.com>
Co-authored-by: Brett Jones <54408245+kowh-ai@users.noreply.github.com>
Co-authored-by: Jari Voutilainen <jari.voutilainen@iki.fi>
Co-authored-by: avdata99 <andres@data99.com.ar>
Co-authored-by: shashi gharti <shashi.gharti@gmail.com>
Co-authored-by: Francesco Frassinelli <francesco.frassinelli@nina.no>
Co-authored-by: Patricio Del Boca <patriciodelboca@gmail.com>
Co-authored-by: Evert Ramos <evert.ramos@gmail.com>
Co-authored-by: Jari Voutilainen <jari-pekka.voutilainen@gofore.com>
Co-authored-by: Rukshan Ranatunge <arkruka@gmail.com>
Co-authored-by: Sergey Motornyuk <sergey.motornyuk@linkdigital.com.au>
Co-authored-by: Konstantin Sivakov <konstantin.sivakov@gmail.com>
Co-authored-by: JVickery-TBS <jesse.vickery@tbs-sct.gc.ca>
Co-authored-by: Jesse Vickery <97247789+JVickery-TBS@users.noreply.github.com>
Co-authored-by: Ben Soroos <bensoroos@derilinx.com>
Co-authored-by: Brett <brett@kowh.ai>

.github/workflows/ckan-docker-image-daily.yml

@@ -1,4 +1,4 @@
-name: CKAN Docker Image
+name: CKAN Docker Prod Image
 
 on:
   workflow_dispatch:

.github/workflows/ckan-docker-image-dev.yml

@@ -1,4 +1,4 @@
-name: CKAN Docker Image
+name: CKAN Docker Dev Image
 
 on:
   workflow_dispatch:
@@ -51,18 +51,38 @@ jobs:
       if: github.event_name == 'release'
       run: echo "TAG=${{ github.event.release.tag_name }}" >> $GITHUB_ENV
 
+    - name: Set up Docker Buildx
+      uses: docker/setup-buildx-action@v3
+
     - name: Login to DockerHub
-      uses: docker/login-action@v2
+      uses: docker/login-action@v3
       with:
         username: ${{ secrets.DOCKER_USERNAME }}
         password: ${{ secrets.DOCKER_PASSWORD }}
-    - uses: satackey/action-docker-layer-caching@v0.0.11
-      # Ignore the failure of a step and avoid terminating the job.
-      continue-on-error: true
-    - run: docker build -t cioos/ckan:DEV_$TAG -t cioos/ckan:DEV_latest .
-    - run: docker push cioos/ckan:DEV_$TAG
-    - if: github.event_name == 'push' || github.event_name == 'workflow_dispatch'
-      run: docker push cioos/ckan:DEV_latest
+
+    - if: github.event_name == 'pull_request'
+      name: Build and push
+      uses: docker/build-push-action@v5
+      with:
+        context: .
+        push: true
+        cache-from: type=gha
+        cache-to: type=gha,mode=max
+        tags: |
+          cioos/ckan:DEV_${{env.TAG}}
+
+    - if: github.event_name != 'pull_request'
+      name: Build and push
+      uses: docker/build-push-action@v5
+      with:
+        context: .
+        push: true
+        cache-from: type=gha
+        cache-to: type=gha,mode=max
+        tags: |
+          cioos/ckan:DEV_latest
+          cioos/ckan:DEV_${{env.TAG}}
+
     - if: github.event_name == 'pull_request'
       uses: mshick/add-pr-comment@v1
       with: