Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create separate nodes for the PPMI and QPN #44

Closed
8 tasks done
Tracked by #36
alyssadai opened this issue Oct 23, 2023 · 5 comments
Closed
8 tasks done
Tracked by #36

Create separate nodes for the PPMI and QPN #44

alyssadai opened this issue Oct 23, 2023 · 5 comments
Assignees
Labels
infrastructure Software or graph infrastructure-related changes.

Comments

@alyssadai
Copy link
Contributor

alyssadai commented Oct 23, 2023

Ideally, these datasets would live on two different different databases (with different APIs) in the same graph server.

In order to allow two different APIs to talk to the same graph container, we may need to create a new docker-compose.yml or otherwise connect one of the API container to an existing docker network (see https://docs.docker.com/engine/reference/commandline/network_connect/).

Steps to implement

  • Launch GraphDB container on st-viateur and add two new PD databases (mni_ppmi and mni_qpn)
  • Launch api_qpn along with graph stack in non-aggregated aggregate mode
  • Launch api_ppmi in aggregate mode
  • Add api_ppmi to api_qpn Docker network (arbitrary) and rename network
    • verify via interactive docs that api_ppmi can reach a db in the graph container
  • for easier relaunching, create a custom docker-compose.yml for the PD databases
    • under ~/projects/pd_compose
    • values for non-API variables hardcoded in the yml
  • Ensure both are accessible without vpn
    • Update security rules for st-viateur to allow ingress on each new API's port
@alyssadai alyssadai changed the title Create a node for the more constrained PD data (API mode) Create separate nodes for the PPMI and QPN Oct 23, 2023
@alyssadai alyssadai added the flag:discuss Flag issue that needs to be discussed before it can be implemented. label Oct 23, 2023
@alyssadai alyssadai moved this to Backlog in Neurobagel Oct 23, 2023
@alyssadai alyssadai moved this from Backlog to Specify - Active in Neurobagel Oct 23, 2023
@alyssadai alyssadai added infrastructure Software or graph infrastructure-related changes. and removed flag:discuss Flag issue that needs to be discussed before it can be implemented. labels Oct 23, 2023
@alyssadai alyssadai moved this from Specify - Active to Specify - Done in Neurobagel Oct 23, 2023
@alyssadai alyssadai moved this from Specify - Done to Implement - Active in Neurobagel Oct 23, 2023
@alyssadai alyssadai self-assigned this Oct 23, 2023
@alyssadai
Copy link
Contributor Author

For reviewer, see on st-viateur:

  • database-specific .env files and docker-compose.yml at ~/projects/pd_compose
  • contents of docker network pd_compose_default (default name based on compose config location) using:
    docker network inspect pd_compose_default
  • newly created PD GraphDB databases, with curl -u "admin:<PASSWORD>" http://localhost:7200/repositories

@alyssadai alyssadai moved this from Implement - Active to Implement - Done in Neurobagel Oct 24, 2023
@surchs surchs moved this from Implement - Done to Review - Active in Neurobagel Oct 25, 2023
@surchs
Copy link
Contributor

surchs commented Oct 25, 2023

Thanks @alyssadai! All config files look great, excellent choice of credentials. However I am getting error 403 from the QPN API:

{"detail":": Forbidden (HTTP status 403)\n"}

I'm assuming that's not what we want?

@alyssadai
Copy link
Contributor Author

alyssadai commented Oct 26, 2023

Thanks for catching that @surchs.

I've re-granted RW permissions to both the mni_qpn and mni_ppmi databases for the database user I set up. If you try to access the QPN API again it should now work.

I've also opened an issue to document the (problem of) non-additive GraphDB permissions here neurobagel/documentation#116.

@alyssadai
Copy link
Contributor Author

Hey @surchs, following recent discussions about the privacy of QPN I've updated the API config for the mni_qpn database to run in aggregate mode as well.

Let me know if this issue looks good to close now 🙂

@github-project-automation github-project-automation bot moved this from Review - Active to Review - Done in Neurobagel Oct 26, 2023
@alyssadai alyssadai reopened this Oct 26, 2023
@github-project-automation github-project-automation bot moved this from Review - Done to Backlog in Neurobagel Oct 26, 2023
@alyssadai alyssadai moved this from Backlog to Review - Active in Neurobagel Oct 26, 2023
@surchs
Copy link
Contributor

surchs commented Oct 27, 2023

@alyssadai I checked the requests and it now works for me with no auth error! Thanks a lot. I think we can close

@github-project-automation github-project-automation bot moved this from Review - Active to Review - Done in Neurobagel Oct 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
infrastructure Software or graph infrastructure-related changes.
Projects
Archived in project
Development

No branches or pull requests

2 participants