Schema package management system based on DOIs and Zenodo

[cmungall]

Writing up some discussion with @hsolbrig

We would like to have a schema package management system such that

• it is easy to install specific versions of schemas that are depended on
• local copies can be made
• everything is archived
• it is easy to make releases
• etc

We would rather avoid the overhead of writing lots of custom software and maintaining a server

note this discussion is about schema package management. for the code we of course use pypi. However it is worth noting that it would be possible to piggy back schema package managent off the back of npm, pypi, etc, as they all allow distribution of files alongside code.

one idea is to use zenodo

• different versions would be under a shared conceptdoi for each package
• there are multiple ways to publish to zenodo; the easiest is just to sync with github https://zenodo.org/account/settings/github/ -- then each release us auto-distributed
• there are various ways for the PM system to access schemas and versions
  • zenodo REST API (no api key required for read operations)
  • use a OAI­PMH harvester

LinkML community on zenodo: https://zenodo.org/communities/linkml?page=1&size=20

this can be queried:

~ oai-harvest https://zenodo.org/oai2d -s "user-linkml" -d datacite3
INFO     Harvesting from https://zenodo.org/oai2d
DEBUG    Writing to file /Users/cjm/datacite3/oai:zenodo.org:6462491.oai_dc.xml
DEBUG    Writing to file /Users/cjm/datacite3/oai:zenodo.org:6463614.oai_dc.xml
DEBUG    Writing to file /Users/cjm/datacite3/oai:zenodo.org:6323601.oai_dc.xml
DEBUG    Writing to file /Users/cjm/datacite3/oai:zenodo.org:6502644.oai_dc.xml
DEBUG    Writing to file /Users/cjm/datacite3/oai:zenodo.org:5598315.oai_dc.xml
DEBUG    Writing to file /Users/cjm/datacite3/oai:zenodo.org:5889549.oai_dc.xml
DEBUG    Writing to file /Users/cjm/datacite3/oai:zenodo.org:6050640.oai_dc.xml
DEBUG    Writing to file /Users/cjm/datacite3/oai:zenodo.org:6321590.oai_dc.xml
DEBUG    Writing to file /Users/cjm/datacite3/oai:zenodo.org:6499627.oai_dc.xml

so a PM system could (given association between package names and concept DOIs) handle downloading latest or all versions of a package

zenodo will be around for a while

https://about.zenodo.org/infrastructure/