Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update api readme #178

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Update api readme #178

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f590fd1
Update api readme
liviuba Sep 3, 2024
a0961a3
Readme fixes
liviuba Sep 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 31 additions & 60 deletions api/Readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,29 +1,5 @@
### First time setup
## Running the api as a service locally

Run
```
poetry install
```
Will generate the following warning:
```
Warning: The current project could not be installed: No file/folder found for package bia-integrator-api
If you do not want to install the current project use --no-root.
If you want to use Poetry only for dependency management but not for packaging, you can set the operating mode to "non-package" in your pyproject.toml file.
In a future version of Poetry this warning will become an error!
```
This can be ignored.

### Install recommended extensions

If doing any development on the project, consider installing the recommended extensions:

* ms-python.black-formatter
* ms-python.debugpy
* ms-python.python


## Running the api

```sh
# note the --build, otherwise the api image doesn't actually get rebuilt to reflect changes
docker compose --env-file ./.env_compose up --build -d # remove -d when first setting up, to make any problems obvious
Expand All @@ -40,54 +16,47 @@ curl -H "Content-Type: application/json" \
http://localhost:8080/v2/auth/user/register
```

The response should be just `null` and there should be no errors in the api container.

## Development
You should be able to authenticate using the email/password above at http://localhost:8080/docs (Authorize button)

### Testing with VS CODE
## First time setup - api dev

The instructions below assume you use VS Code for development.
⚠️ Note that unless you run uvicorn on the host, the api at http://localhost:8080 is the one in the container (above) - so rebuild when needed

For test/debugger integration to work, **the api directory must be the root project directory in vscode**, not the bia-integrator directory.
⚠️ It's simplest to never run the api on the host, to avoid port clashes/confusion between it and the one running in docker. For API testing (pytest cli or vscode pytest plugin) the api never listens on any port. If you need a local api, use the one in docker (and rebuild / every time you make changes)

In order to run tests, the database needs to be up, which you can do by running the docker compose command listed above.
⚠️ To cleanup the local mongo `docker compose down`

You also need to have a `.env` file under the `/api/` folder. This configures the API that is tested to be able to access your locally running database. This .env file should not be committed to the package, as it would interact with the production service. You can copy & fill in the `.env_template` and use the `.env_compose` for reference.
⚠️ **Only interacting with the api through tests is recommended and documented here** If using the api as a client, see the [api client](../clients/python/). For one-off requests use http://localhost:8080/docs .

### Adding new tests

Tests need to be created in a file starting with `test_` under the `tests` folder. They should be functions that start with `test_` e.g `def test_study_creation()` in the `test_study.py` file. This will allow vscode to identify them.
Follow steps [above](#running-the-api-as-a-service-locally) to get a mongo instance and an api running.

### Updating API code while testing.
For VSCode:
* Recommended extensions: ms-python.black-formatter, ms-python.debugpy, ms-python.python
* `bia-integrator/api` should be the project root (opened in vscode) for plugins to work smoothly
* `poetry install --no-root` create a virtualenv with api dependencies. Set the python binary in VSCode to the python there (in a .py file, lower-right)
* **Testing** tab should collect the tests and be able to run them. **Debugger** should work (set a breakpoint and run 'Debug test' to test)

Can perform some on-the-fly testing by running the following commands:
### Setup & run tests only

Build images and run contains:
Follow steps [above](#running-the-api-as-a-service-locally) to get a mongo instance running

`docker compose --env-file ./.env_compose up --build -d`

Create user:

`curl -H "Content-Type: application/json" --request POST --data '{"email": "test@example.com", "password_plain": "test", "secret_token": "0123456789==" }' http://localhost:8080/v2/auth/user/register`

Get auth token

`curl -H "Content-Type: application/x-www-form-urlencoded" --request POST --data 'username=test@example.com&password=test' http://localhost:8080/v2/auth/token`

Copy auth token which you can then use to make calls to the api. E.g. create a study:

`curl -H "Content-Type: application/json" -H "Authorization: Bearer <auth token>" --request POST http://localhost:8080/v2/private/study -d @study_input.json`

You should be able to make your changes & can rebuild the api image with the command:
```sh
cp .env_compose .env

`docker build -t api-bia-integrator-api:latest .`
## change (at least) the mongo host in MONGO_CONNSTRING to be localhost instead of biaint-mongo
vim .env

And then update the running container with your code.
# should be in bia-integrator/api
pwd

`docker compose up -d`
# the api used for testing runs on the host (not in docker) and is configures with the .env created above
# ! poetry auto-loads .env
poetry run pytest
```

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Below worked fine until
docker build -t api-bia-integrator-api:latest .
Got the following error:

[+] Building 2.1s (10/13) docker:desktop-linux
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.01kB 0.0s
=> [internal] load metadata for docker.io/library/python:3.10.0 1.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [1/9] FROM docker.io/library/python:3.10.0@sha256:a1fba384d3aa879533a271ab5d73965fbd3385648f38835dacd07e519f1c7c3f 0.0s
=> [internal] load build context 1.0s
=> => transferring context: 628.46kB 1.0s
=> CACHED [2/9] RUN curl -sSL https://install.python-poetry.org | python3 - 0.0s
=> CACHED [3/9] WORKDIR /bia-integrator 0.0s
=> [4/9] ADD api/poetry.lock* api/poetry.lock 0.0s
=> ERROR [5/9] ADD api/pyproject.toml api/pyproject.toml 0.0s
=> ERROR [6/9] ADD bia-shared-datamodels bia-shared-datamodels 0.0s

[5/9] ADD api/pyproject.toml api/pyproject.toml:

[6/9] ADD bia-shared-datamodels bia-shared-datamodels:

Dockerfile:17

15 |
16 | # This needed to be added because it's a poetry dep
17 | >>> ADD bia-shared-datamodels bia-shared-datamodels
18 |
19 | WORKDIR /bia-integrator/api

ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref 4a7e193a-3bcc-4f93-b2be-9c4f5ccebe46::0hkxb8pimlv2bx4oy73ozdx1h: "/bia-shared-datamodels": not found

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was this run in bia-integrator/api? Adding that in the readme now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nvm, I thought that part was still working so left it in but it's not because we switched context root. Fixing now

Alternatively, if using VS Code, you can open the code that is running in the docker container and make changes there directly.
### Adding new tests

Tests need to be created in a file starting with `test_` under the `tests` folder. They should be functions that start with `test_` e.g `def test_study_creation()` in the `test_study.py` file. This will allow vscode to identify them.

### Pushing to docker hub image registry

Expand All @@ -97,9 +66,11 @@ To rebuild the api image and push it to the docker container registry.
# Create a github personal access token here: https://github.com/settings/tokens
# with write:packages scope

# building the api image
# !! should be in bia-integrator, not bia-integrator/api
pwd

docker login ghcr.io
docker build -t bioimage-archive/integrator-api:0.1 .
docker build -f api/Dockerfile -t bioimage-archive/integrator-api:0.1 .
docker image tag bioimage-archive/integrator-api:0.1 ghcr.io/bioimage-archive/integrator-api:0.1
docker push ghcr.io/bioimage-archive/integrator-api:0.1
```
Expand Down
Loading