From 993bbf4e8870114fe19837af3bde500e7f976c60 Mon Sep 17 00:00:00 2001 From: xnought Date: Thu, 22 Feb 2024 19:42:51 -0800 Subject: [PATCH 1/9] feat: change intro docs --- DEVELOPMENT.md | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 2d30d88c..4daa894a 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -1,12 +1,14 @@ # Development +If you want to run the frontend and backend yourself, you can! Keep reading... + By the time you finish this guide, you will be able to run the JS frontend [Svelte](https://kit.svelte.dev/) and the Python backend [FastAPI](https://fastapi.tiangolo.com/). Note that we use `docker-compose` as defined in the [`docker-compose.yml`](./docker-compose.yml). If you want to see more specific docs go to the [ `docs/` ](./docs/README.md) for more info. -## Installation +## Quick Start > [!IMPORTANT] > You must have [Docker Desktop](https://www.docker.com/products/docker-desktop/) GUI installed and the `docker-compose` bash command. @@ -17,18 +19,18 @@ You can run everything by doing sh run.sh ``` -and navigate to [http://0.0.0.0:5173](http://0.0.0.0:5173) webserver (or [http://localhost:5173](http://localhost:5173)) - -That's it. This will spin up a docker container with the backend, database, and frontend servers running. - -To turn it off do +or directly executing ```bash -sh run.sh stop +./run.sh ``` -Check out the [`run.sh`](./run.sh) for more shortcuts like this. -If you want intellisense/autocomplete in VSCode, continue to the rest of the installation. Otherwise you are done. Totally optional, but I'd do it for autocomplete. +this will download a docker container running the svelte frontend, the python backend, and the postgres database all in development hot-reload mode. Installation may take a few minutes. + +Now navigate to [http://localhost:5173](http://localhost:5173) to see the frontend. + +> [!TIP] +> Check the [`docs/run.md`](./docs/run.md) for documentation on the [`run.sh`](./run.sh) file for more build commands ### Backend From 84c7f0222592b31e0040c67b2e3f0acb33af781e Mon Sep 17 00:00:00 2001 From: xnought Date: Thu, 22 Feb 2024 19:49:56 -0800 Subject: [PATCH 2/9] feat: simplify the docs development --- DEVELOPMENT.md | 86 ++++++++++---------------------------------------- 1 file changed, 17 insertions(+), 69 deletions(-) diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 4daa894a..98d366f3 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -32,12 +32,18 @@ Now navigate to [http://localhost:5173](http://localhost:5173) to see the fronte > [!TIP] > Check the [`docs/run.md`](./docs/run.md) for documentation on the [`run.sh`](./run.sh) file for more build commands -### Backend +## Local Development Environment -> [!IMPORTANT] -> Before you do anything, make sure you have [Python](https://www.python.org/downloads/) installed. +We use VSCode for all development and make sure to download a few extensions like + +- Ruff code formatter (for backend) +- Prettier code formatter (for frontend) +- Svelte (for frontend) +- Python (for backend) + +for a better experience. Then to get autocomplete with those languages, you'll need to manually install the packages that have already been installed in the docker, but locally. Otherwise your VSCode won't know how to autocomplete. -Then install the [Poetry Python Package Manager](https://python-poetry.org/) by doing +First install the [Poetry Python Package Manager](https://python-poetry.org/) by doing ```bash pip3 install poetry @@ -50,78 +56,20 @@ poetry config virtualenvs.in-project true # required for the .venv to get create poetry install ``` -**๐Ÿฅณ You're done with the backend installation!** - -### Frontend - -> [!IMPORTANT] -> Before you do anything, make sure you first have [Node](https://nodejs.org/en/download) installed. - -If you haven't already, also install [Yarn](https://classic.yarnpkg.com/lang/en/docs/install/#mac-stable) globally as the package manager for node by doing - -```bash -npm install --global yarn -``` - -Nice you have Yarn ๐Ÿงถ now! Now install the frontend packages by doing +Then install frontend packages ```bash cd frontend -yarn install +yarn ``` -**๐Ÿฅณ You're done with the frontend installation!** - - -## Navigating the codebase - -> [!WARNING] -> Ignore the build script in the video, they are outdated -> We use docker now through `sh run.sh` -> Additionally the host has changed from `localhost` to `0.0.0.0` - -I'll explain top-down how to understand the structure of this repository. First of all, you want to mainly pay attention to the [`frontend`](./frontend/) then the [`backend`](./backend/) code. I'll explain: - -### Frontend - -> [!IMPORTANT] -> ๐ŸŽฅ [Click here](https://drive.google.com/file/d/1KD3Hgbul0_7cIZiCaQZ1U4meCBthcrfS/view?usp=drive_link) to watch the frontend video overview by @xnought. - - -The main code for frontend is in [`frontend/src`](./frontend/src). We use the JavaScript framework called [Svelte](https://svelte.dev/) which makes writing frontends super easy and is the standard for websites with visualizations. To be very specific, we use [SvelteKit](https://kit.svelte.dev/) which is a framework built on the Svelte compiler for websites with multiple routes. - -The home page you see when you run the frontend server is defined in [`frontend/src/routes/+page.svelte`](./frontend/src/routes/+page.svelte). - -To make another route, for example an `/about` to show an about page in the frontend, we just create a directory with another `+page.svelte`. - - -For example, [`frontend/src/routes/about/+page.svelte`](./frontend/src/routes/about/+page.svelte). - -See SvelteKit documentation for more info on how to add more complicated behavior. There are a ton of nice things to loop over data directly in the DOM, insert reactive variables, and maintain reactive global state. Ask @xnought if you want to learn more. - -### Backend - -> [!IMPORTANT] -> ๐ŸŽฅ [Click here](https://drive.google.com/file/d/1mmZqsALCY4UhcT592GR0Q1PMtZPogXkq/view?usp=drive_link) to watch the backend video overview by @xnought. - -The frontend calls to the backend. The backend is in Python3 and specifically [FastAPI](https://fastapi.tiangolo.com/). FastAPI is a nice HTTP REST server where we can make HTTP requests to the backend from the frontend. +## Where to look -The main server lives in [`backend/src/server.py`](./backend/src/server.py). +I'd recommend going to the [`frontend/package.json`](./frontend/package.json) and [`frontend/src/App.svelte`](./frontend/src/App.svelte) if you want to get started in the frontend. -Check out the FastAPI docs for more info. +I'd recommend going to the [`backend/src/server.py`](./backend/src/server.py) and [`backend/init.sql`](./backend/init.sql) for the backend/db. -### Frontend and Backend Interaction -> [!IMPORTANT] -> ๐ŸŽฅ [Click here](https://drive.google.com/file/d/1micYztZj8q5oufOhVctzPvsE9U1NgnyS/view?usp=drive_link) to watch how to interact with the backend from the frontend -Once we created the HTTP endpoints in the backend, we can call -```bash -sh run.sh api -``` - -which generates frontend bindings/functions we can use in the frontend. See the [`+page.svelte`](./frontend/src/routes/+page.svelte) for an example usage with the `Backend` object (generated from the command above). You'll see in the network tab that it just makes a `fetch` call to the backend. - -### Database - -We use PostgreSQL in Docker. For more information go to the docs at [`database.md`](./docs/database.md). +> [!TIP] +> If you have more questions, create an issue on this github or look for specific documentation in the [`docs/`](./docs/) directory From e148d926038f702c3c3a5cf68b7fd3e5931f1bd5 Mon Sep 17 00:00:00 2001 From: xnought Date: Mon, 26 Feb 2024 20:33:11 -0800 Subject: [PATCH 3/9] docs: update run.md --- docs/run.md | 62 +++++++++++++++++++++++++++++++++------------- frontend/README.md | 47 ----------------------------------- run.sh | 11 -------- 3 files changed, 45 insertions(+), 75 deletions(-) delete mode 100644 frontend/README.md diff --git a/docs/run.md b/docs/run.md index e7c38245..bce674f4 100644 --- a/docs/run.md +++ b/docs/run.md @@ -1,6 +1,6 @@ # [`run.sh`](../run.sh) -The [`run.sh`](../run.sh) contains all the commands you would ever want to run. +The [`run.sh`](../run.sh) contains all the commands to run the backend, frontend, restart the containers, and more. You can easily call a function defined in the [`run.sh`](../run.sh) by calling @@ -8,27 +8,55 @@ You can easily call a function defined in the [`run.sh`](../run.sh) by calling sh run.sh ``` -`` can be any one of these useful commands (I noted ๐Ÿ‘‰ as the most useful/important in my opinion) +or + +```bash +./run.sh +``` + +## Common Uses + +**starting up** + +```bash +./run.sh start +``` + +**If you're having big errors and have tried everything. Start over from scratch with** + +```bash +./run.sh hard_restart +``` + +**If you want to restart everything without nuking the whole project, try a softer version** + + +```bash +./run.sh soft_restart +``` + + +**If you changed the [`init.sql`](../backend/init.sql) and you're not seeing any changes** + +```bash +./run.sh reload_init_sql +``` + +## More Commands | `` | what it does | |---|---| -| `start` | ๐Ÿ‘‰ Starts/runs the docker container | -| `stop` | ๐Ÿ‘‰ Stops the docker container | -| `refresh_packages` | ๐Ÿ‘‰ Updates or adds new packages in frontend and backend, needs to be run every time you add a new package locally | -| `restart` | Stops, then Starts the docker container | -| `soft_restart` | ๐Ÿ‘‰ Updates or adds packages, reloads the init sql, and restarts. Use this if you're getting errors. | -| `hard_restart` | ๐Ÿ‘‰ Builds the docker container from scratch and starts it, do this if you're having errors that even `soft_restart` can't fix | -| `gen_api` | ๐Ÿ‘‰ Creates/Generates the frontend api code based on the backend endpoints | -| `reload_init_sql` | ๐Ÿ‘‰ When you change the the schema in [`init.sql`](../backend/init.sql), this reloads it, otherwise your changes won't show up in the docker | +| `start` | Starts/runs the docker container where edits to the backend or frontend will hot reload | +| `stop` | Stops the docker container | +| `soft_restart` | Updates or adds packages, reloads the database from [`init.sql`](../backend/init.sql). | +| `hard_restart` | Builds the docker container from scratch and starts it, do this if you're having errors that even `soft_restart` can't fix | +| `refresh_packages` | Updates or adds new packages in frontend and backend, needs to be run every time you add a new package locally | +| `gen_api` | Creates/Generates the frontend api code based on the backend endpoints | +| `reload_init_sql` | When you change the the schema in [`init.sql`](../backend/init.sql), this reloads it, otherwise your changes won't show up in the docker | | `sql_dump` | Dumps the current db contents and schema into [`backups/`](../backend/backups/).| | `psql` | Opens up a direct terminal into the database to execute SQL commands live | -| `test` | Runs all unit tests | -| `test_backend` | Runs only backend unit tests | -| `test_frontend` | Runs only frontend unit tests | -| `upload_all` | ๐Ÿ‘‰ Uploads all the pdb files to the system via POST requests | -| `delete_all` | Deletes all protein entries and restarts the server from scratch | +| `upload_all` | Uploads all the pdb files to the system via POST requests | +| `delete_all` | Deletes all protein entries and restarts the server from scratch | There are actually many more functions, so please check out [`run.sh`](../run.sh). - - diff --git a/frontend/README.md b/frontend/README.md deleted file mode 100644 index e6cd94fc..00000000 --- a/frontend/README.md +++ /dev/null @@ -1,47 +0,0 @@ -# Svelte + TS + Vite - -This template should help get you started developing with Svelte and TypeScript in Vite. - -## Recommended IDE Setup - -[VS Code](https://code.visualstudio.com/) + [Svelte](https://marketplace.visualstudio.com/items?itemName=svelte.svelte-vscode). - -## Need an official Svelte framework? - -Check out [SvelteKit](https://github.com/sveltejs/kit#readme), which is also powered by Vite. Deploy anywhere with its serverless-first approach and adapt to various platforms, with out of the box support for TypeScript, SCSS, and Less, and easily-added support for mdsvex, GraphQL, PostCSS, Tailwind CSS, and more. - -## Technical considerations - -**Why use this over SvelteKit?** - -- It brings its own routing solution which might not be preferable for some users. -- It is first and foremost a framework that just happens to use Vite under the hood, not a Vite app. - -This template contains as little as possible to get started with Vite + TypeScript + Svelte, while taking into account the developer experience with regards to HMR and intellisense. It demonstrates capabilities on par with the other `create-vite` templates and is a good starting point for beginners dipping their toes into a Vite + Svelte project. - -Should you later need the extended capabilities and extensibility provided by SvelteKit, the template has been structured similarly to SvelteKit so that it is easy to migrate. - -**Why `global.d.ts` instead of `compilerOptions.types` inside `jsconfig.json` or `tsconfig.json`?** - -Setting `compilerOptions.types` shuts out all other types not explicitly listed in the configuration. Using triple-slash references keeps the default TypeScript setting of accepting type information from the entire workspace, while also adding `svelte` and `vite/client` type information. - -**Why include `.vscode/extensions.json`?** - -Other templates indirectly recommend extensions via the README, but this file allows VS Code to prompt the user to install the recommended extension upon opening the project. - -**Why enable `allowJs` in the TS template?** - -While `allowJs: false` would indeed prevent the use of `.js` files in the project, it does not prevent the use of JavaScript syntax in `.svelte` files. In addition, it would force `checkJs: false`, bringing the worst of both worlds: not being able to guarantee the entire codebase is TypeScript, and also having worse typechecking for the existing JavaScript. In addition, there are valid use cases in which a mixed codebase may be relevant. - -**Why is HMR not preserving my local component state?** - -HMR state preservation comes with a number of gotchas! It has been disabled by default in both `svelte-hmr` and `@sveltejs/vite-plugin-svelte` due to its often surprising behavior. You can read the details [here](https://github.com/rixo/svelte-hmr#svelte-hmr). - -If you have state that's important to retain within a component, consider creating an external store which would not be replaced by HMR. - -```ts -// store.ts -// An extremely simple external store -import { writable } from 'svelte/store' -export default writable(0) -``` diff --git a/run.sh b/run.sh index e3f809c2..07a23c90 100755 --- a/run.sh +++ b/run.sh @@ -60,17 +60,6 @@ function test_backend() { cd .. } -function test_frontend() { - cd frontend - yarn test - cd .. -} - -function test() { - test_backend - test_frontend -} - function install_frontend() { docker exec -it venome-frontend yarn install } From d2bdfd7cf5e5b73723a3f2fa4a0a8b009112bea6 Mon Sep 17 00:00:00 2001 From: xnought Date: Mon, 26 Feb 2024 20:44:30 -0800 Subject: [PATCH 4/9] docs: frontend changes --- docs/frontend.md | 54 ++++++++++++++++++++---------------------------- 1 file changed, 22 insertions(+), 32 deletions(-) diff --git a/docs/frontend.md b/docs/frontend.md index af371d16..9fa77100 100644 --- a/docs/frontend.md +++ b/docs/frontend.md @@ -1,8 +1,11 @@ -# Frontend +# [frontend](../frontend/) -In [SvelteKit](https://kit.svelte.dev/) which uses the [Svelte](https://svelte.dev/) JavaScript framework which makes frontend work really freaking easy and great for visualizations. +We use Svelte and specifically vite to build the application. The main svelte code is located in [frontend/src/](../frontend/src/). + +We also have a separate library called [frontend/venome-molstar](../frontend/venome-molstar/) which is a separate project for the visualization system. + +We directly include the source code from venome-molstar in our application so no need to build it separately. -Here are [tutorials](https://learn.svelte.dev/tutorial/welcome-to-svelte) if you are interested, you'll find it simpler than regular JavaScript and much faster than other frameworks like React. ## UI Styling @@ -12,50 +15,37 @@ Check out Flowbite to use those ready-made svelte components (like buttons, drop ## Add packages +First add the package locally for intellisense + ```bash cd frontend yarn add -W js_package_name ``` +(-W is needed for yarn workspaces which is what we are doing) -which adds the package locally (so your intellisense can detect it). - -To have those packages also installed/reflected in the backend run +then install in the docker container with ```bash -sh run.sh refresh_packages +sh run.sh install_frontend ``` -or rebuild the entire docker (this method is slower, but guaranteed to work) with - -```bash -sh run.sh hard_restart -``` +## Backend fetch requests -## Access to Backend Functions +You can simply import the `Backend` object and call the functions by name (which will subsequently call the server). -You can create functions in the [`backend/server.py`](../backend/src/server.py) and import them into the frontend. - -To do this, you need to have saved your changes in the backend, restart the server, then run `./run.sh gen_api` from the project root. - -Now you can simply import the `Backend` object and call the functions by name (which will subsequently call the server). - -Example +Here is an example where I call the backend function found in [backend/src/api/search.py](../backend/src/api/search.py#L115) called `search_range_length` and in it get translated into camel case as ```ts -import { Backend } from '$lib/backend'; -console.log(await Backend.helloWord()) // > {"message": "Hello World"} +import { Backend } from '../lib/backend'; +console.log(await Backend.searchRangeLength()) ``` -It's that easy! +Under the hood the `Backend` object simply does a `fetch()` but here we get type generation and an easy API. -> [!NOTE] -> The `Backend.helloWorld()` corresponds exactly to the backend function defined in the HTTP server. Which is automatically generated by `yarn openapi` based on the backend HTTP endpoints. -> -> Check [`server.py`](../backend/src/server.py) and you'll find `hello_world` as a function too. -> -> If you must know, the frontend uses this interface `Backend.helloWorld()` to make a GET request to the backend at the function `hello_world` in [`server.py`](../backend/src/server.py). Luckily this is abstracted away with `yarn openapi` / `./run.sh api` API generation. +To generate this `Backend` API in the frontend, you must run +```bash +./run.sh gen_api +``` -## Need help? - -First check [SvelteKit](https://kit.svelte.dev/) docs to figure out what files mean what. Then ask @xnought. +which reads from the server and generates that library for the frontend. Super cool! In other words it reads our end points and create a nice wrapper over them for the frontend. From 646de55cd8a0981324a6b15222a934305a4bc761 Mon Sep 17 00:00:00 2001 From: xnought Date: Mon, 26 Feb 2024 20:52:51 -0800 Subject: [PATCH 5/9] feat: documentation toc --- docs/README.md | 7 +-- docs/backend.md | 112 ++++++++++++++++++++++++++++++++++++++++++----- docs/database.md | 98 ----------------------------------------- docs/docker.md | 52 ---------------------- docs/testing.md | 46 ------------------- 5 files changed, 104 insertions(+), 211 deletions(-) delete mode 100644 docs/database.md delete mode 100644 docs/docker.md delete mode 100644 docs/testing.md diff --git a/docs/README.md b/docs/README.md index 4bc1fb41..08f5e360 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,12 +1,9 @@ # Docs -Below are all the documentation for how to do things in this codebase. The only documentation not in this dir. is the [`DEVELOPMENT.md`](../DEVELOPMENT.md) +Below are all the documentation for how to do things in this codebase. The only documentation not in this dir. is the [`DEVELOPMENT.md`](../DEVELOPMENT.md) which has the best chance of helping you get started. **Table of Contents** -- [`database.md`](./database.md) -- [`testing.md`](./testing.md) - [`frontend.md`](./frontend.md) - [`backend.md`](./backend.md) -- [`run.md`](./run.md) -- [`docker.md`](./docker.md) \ No newline at end of file +- [`run.md`](./run.md) \ No newline at end of file diff --git a/docs/backend.md b/docs/backend.md index f01c9cb2..2a3ad2c8 100644 --- a/docs/backend.md +++ b/docs/backend.md @@ -1,15 +1,17 @@ -# Backend +# [backend](../backend/) -These docs go over the [`backend/`](../backend/) dir. Go to the [`database.md`](./database.md) if you were looking for db specific things. +The backend consists of a python HTTP server with FastAPI and a postgreSQL database all housed in a docker container. +## How the HTTP Server works -## Logging +The main file to look at is [backend/src/server.py](../backend/src/server.py) which hosts all our endpoints. Each endpoint can be included via a router found in the [backend/src/api/](../backend/src/api/) folder. -Note that `print` is wonky on docker, so instead import `logging as log` and use `log.warn(stuff)` to print stuff to the console in docker. +See examples in that folder for how we set up get, post, and other types of HTTP requests. -## Add packages +## Adding packages to the python server ```bash +cd backend poetry add py_package_name ``` @@ -18,18 +20,108 @@ which adds the package locally (so your intellisense can detect it). To have those packages also installed/reflected in the backend run ```bash -sh run.sh refresh_packages +sh run.sh install_backend ``` -or rebuild the entire docker (this method is slower, but guaranteed to work) with +## Database + +This documentation goes over how to deal with the PostgreSQL database. + +### Usage + +Then, you can execute SQL queries by + +```py +with Database() as db: + db.execute("""""") +``` + +or if you want the values returned from the execute, use + +```py +with Database() as db: + out = db.execute_return("""""") +``` + +which will return a list of the results too. + +> [!NOTE] +> `with Database() as db` +> opens up the `db` connection, then closes it when the scope is finished + +### Handling Query Errors from Python Backend + +Wrap `execute` or `execute_return` in try except statements like this: + +```py +with Database() as db: + try: + out = db.execute_return("""""") + except Exception as e: + # here if you encounter an error in the above query + pass +``` + +to catch errors. + +### PostgreSQL Shell + +To use the shell to run SQL commands to directly on the database, run ```bash -sh run.sh hard_restart +sh run.sh psql ``` +which will open up the `venome-posgres` docker image and login to `psql` (posgresSQL) for you. + +You should get access to the psql shell now, where you can run stuff like + +Screenshot 2023-11-11 at 4 17 16โ€ฏPM + -## Testing +If you have errors, make sure the docker container is already running (`./run.sh start` cmd). + +### Backups +If you plan to make any glaring changes, make sure to make backups/sql dumps with + +```bash +sh run.sh sql_dump +``` +and verify the backup succeeded by checking the [`backend/backups/`](../backend/backups/README.md). + +### Reloading from the [`init.sql`](../backend/init.sql) + +> [!WARNING] +> Reloading this file will override the existing data and schema in the running db + +The [`init.sql`](../backend/init.sql) only gets loaded on the first run of docker. It stops running this file on subsequent docker runs. + +To reload the docker db with this file, simply run ```bash -sh run.sh test_backend +sh run.sh reload_init_sql ``` +## Docker + +> [!IMPORTANT] +> You must have [Docker Desktop](https://www.docker.com/products/docker-desktop/) GUI installed and the `docker-compose` bash command. + +We use docker to run our environments. We compose three images + +- venome-backend (on port `8000`) +- venome-frontend (on port `5173`) +- venome-postgres (on port `5432`) + +all are defined in the [docker-compose.yml](../docker-compose.yml). + +more commands can be found in the [`run.md`](run.md) and more specific instructions can be found in the [`DEVELOPMENT.md`](../DEVELOPMENT.md). + +### Inspecting the consoles + +While the docker venome container is running, go to the Docker Desktop GUI. You should see the same as the following video + + +https://github.com/xnought/venome/assets/65095341/c44f1d8c-0d58-407c-9aa2-29c4a92fb666 + + +this is where you can see print statements and other debug info / errors. \ No newline at end of file diff --git a/docs/database.md b/docs/database.md deleted file mode 100644 index a6392f25..00000000 --- a/docs/database.md +++ /dev/null @@ -1,98 +0,0 @@ -# Database - -This documentation goes over how to deal with the PostgreSQL database. - -## Query from Python Backend - -There are some real examples of db use in the [`server.py`](../backend/src/server.py). - -First, import the `Database` class from [`db.py`](../backend/src/db.py). - -### Usage - - -Then, you can execute SQL queries by - -```py -with Database() as db: - db.execute("""""") -``` - -or if you want the values returned from the execute, use - -```py -with Database() as db: - out = db.execute_return("""""") -``` - -which will return a list of the results too. - -> [!NOTE] -> `with Database() as db` -> opens up the `db` connection, then closes it when the scope is finished - -### Handling Query Errors from Python Backend - -Wrap `execute` or `execute_return` in try except statements like this: - -```py -with Database() as db: - try: - out = db.execute_return("""""") - except Exception as e: - # here if you encounter an error in the above query - pass -``` - -to catch errors. - -## PostgreSQL Shell - -To use the shell to run SQL commands to directly on the database, run - -```bash -sh run.sh psql -``` -which will open up the `venome-posgres` docker image and login to `psql` (posgresSQL) for you. - -You should get access to the psql shell now, where you can run stuff like - -Screenshot 2023-11-11 at 4 17 16โ€ฏPM - - -If you have errors, make sure the docker container is already running (`start` cmd). - -## Backups -If you plan to make any glaring changes, make sure to make backups/sql dumps with - -```bash -sh run.sh sql_dump -``` -and verify the backup succeeded by checking the [`backend/backups/`](../backend/backups/README.md). - -## Reloading from the [`init.sql`](../backend/init.sql) - -> [!WARNING] -> Reloading this file will override the existing data and schema in the running db - -The [`init.sql`](../backend/init.sql) only gets loaded on the first run of docker. It stops running this file on subsequent docker runs. - -To reload the docker db with this file, simply run - -```bash -sh run.sh reload_init_sql -``` - -## Changes to the Schema/Tables - -**If you DON'T care about losing data**, consider simply editing the [`init.sql`](../backend/init.sql) with the updated schema, and reloading the db with this file as outlined in the [Previous Section](#reloading-from-the-initsql). - -> [!IMPORTANT] -> As of now, we don't care about losing user data since we haven't deployed the system. -> The following is a problem for future us. - -**If you DO care about losing data** we need to be more careful. Consider making a [Backup](#backups) first. Then doing the previous method. Then you can manually copy over the insert statements from the dump into the init.sql until we figure out a better way to make modifications. - -**Shell Method** You can also make alterations to the current table and schema with [modifications](https://www.postgresql.org/docs/current/ddl-alter.html) on the `venome-postgres` psql shell which I showed you how to access in the [PostgreSQL Shell](#postgresql-shell) previous section. You will need to make backups and reinsert these changes into the init.sql so everyone else also gets these alterations. - - diff --git a/docs/docker.md b/docs/docker.md deleted file mode 100644 index 2af67f65..00000000 --- a/docs/docker.md +++ /dev/null @@ -1,52 +0,0 @@ -# Docker - - -> [!IMPORTANT] -> You must have [Docker Desktop](https://www.docker.com/products/docker-desktop/) GUI installed and the `docker-compose` bash command. - -We use docker to run our environments. We compose three images - -- venome-backend (on port `8000`) -- venome-frontend (on port `5173`) -- venome-postgres (on port `5432`) - -all are defined in the [docker-compose.yml](../docker-compose.yml). - -> [!NOTE] -> The postgres db is available outside of the container (your computer) on port `3000` instead. The ports above are internal docker container ports. - -## Getting Started - -To start the container - -```bash -sh run.sh start -``` - -now you can navigate to [http://localhost:5173](http://localhost:5173) to see the interface running. - -and to stop the container - -```bash -sh run.sh stop -``` - -to rebuild the container from scratch run - -```bash -sh run.sh hard_restart -``` - -more commands can be found in the [`run.md`](run.md) and more specific instructions can be found in the [`DEVELOPMENT.md`](../DEVELOPMENT.md), [`backend.md`](./backend.md), or [`database.md`](./database.md). - -## Inspecting the consoles - -While the docker venome container is running, go to the Docker Desktop GUI. You should see the same as the following video - - -https://github.com/xnought/venome/assets/65095341/c44f1d8c-0d58-407c-9aa2-29c4a92fb666 - - -this is where you can see print statements and other debug info / errors. - - diff --git a/docs/testing.md b/docs/testing.md deleted file mode 100644 index b0adb969..00000000 --- a/docs/testing.md +++ /dev/null @@ -1,46 +0,0 @@ -# Testing - -Tests will run on pull request, but here are some docs on how to test locally or add new tests. - -## Running Tests - -You can run tests with - -```bash -sh run.sh test -``` - -which runs the backend and frontend tests. - - -To run them individually by - -### Run Frontend Tests - -```bash -sh run.sh test_frontend -``` - -### Run Backend Tests - -```bash -sh run.sh test_backend -``` - -## Creating Tests - -### Creating Frontend Tests - -In the frontend add your tests to [`frontend/src/tests`](../frontend/src/tests/) folder. - -Specifically name your files with `filename.test.ts` to label them as test files. Then check out the existing examples to see how to add unit tests. - -Checkout [`example.test.ts`](../frontend/src/tests/example.test.ts) for an example. - -### Creating Backend Tests - -In the backend add your tests to [`backend/src/tests`](../backend/src/tests/) folder. - -Specifically name your files with `filename_test.py` to label them as test files. Then check out the existing examples to see how to add unit tests. - -Checkout [`example_test.py`](../backend/src/tests/example_test.py) for an example. From 69a22c44bbe461899cf7d3e893df254950b15c5b Mon Sep 17 00:00:00 2001 From: xnought Date: Mon, 26 Feb 2024 20:56:29 -0800 Subject: [PATCH 6/9] consolidate --- DEVELOPMENT.md | 75 ---------------------------------------------- README.md | 81 +++++++++++++++++++++++++++++++++++++++++++++----- 2 files changed, 73 insertions(+), 83 deletions(-) delete mode 100644 DEVELOPMENT.md diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md deleted file mode 100644 index 98d366f3..00000000 --- a/DEVELOPMENT.md +++ /dev/null @@ -1,75 +0,0 @@ -# Development - -If you want to run the frontend and backend yourself, you can! Keep reading... - -By the time you finish this guide, you will be able to run the JS frontend [Svelte](https://kit.svelte.dev/) and the Python backend [FastAPI](https://fastapi.tiangolo.com/). - -Note that we use `docker-compose` as defined in the [`docker-compose.yml`](./docker-compose.yml). - -If you want to see more specific docs go to the [ `docs/` ](./docs/README.md) for more info. - -## Quick Start - -> [!IMPORTANT] -> You must have [Docker Desktop](https://www.docker.com/products/docker-desktop/) GUI installed and the `docker-compose` bash command. - -You can run everything by doing - -```bash -sh run.sh -``` - -or directly executing - -```bash -./run.sh -``` - -this will download a docker container running the svelte frontend, the python backend, and the postgres database all in development hot-reload mode. Installation may take a few minutes. - -Now navigate to [http://localhost:5173](http://localhost:5173) to see the frontend. - -> [!TIP] -> Check the [`docs/run.md`](./docs/run.md) for documentation on the [`run.sh`](./run.sh) file for more build commands - -## Local Development Environment - -We use VSCode for all development and make sure to download a few extensions like - -- Ruff code formatter (for backend) -- Prettier code formatter (for frontend) -- Svelte (for frontend) -- Python (for backend) - -for a better experience. Then to get autocomplete with those languages, you'll need to manually install the packages that have already been installed in the docker, but locally. Otherwise your VSCode won't know how to autocomplete. - -First install the [Poetry Python Package Manager](https://python-poetry.org/) by doing - -```bash -pip3 install poetry -``` - -Then install the packages listed under [`pyproject.toml`](./backend/pyproject.toml) by doing - -```bash -poetry config virtualenvs.in-project true # required for the .venv to get created -poetry install -``` - -Then install frontend packages - -```bash -cd frontend -yarn -``` - -## Where to look - -I'd recommend going to the [`frontend/package.json`](./frontend/package.json) and [`frontend/src/App.svelte`](./frontend/src/App.svelte) if you want to get started in the frontend. - -I'd recommend going to the [`backend/src/server.py`](./backend/src/server.py) and [`backend/init.sql`](./backend/init.sql) for the backend/db. - - - -> [!TIP] -> If you have more questions, create an issue on this github or look for specific documentation in the [`docs/`](./docs/) directory diff --git a/README.md b/README.md index 1e99b0f5..568e6f91 100644 --- a/README.md +++ b/README.md @@ -7,20 +7,85 @@ A wikipedia ๐Ÿ“– for venom proteins: upload, search, organize, visualize, and do ๐Ÿค In collaboration with the ๐Ÿงช[Venom Biochemistry & Molecular Biology Laboratory](https://venombiochemistrylab.weebly.com/) at Oregon State University ๐Ÿฆซ. > [!IMPORTANT] -> - [`DEVELOPMENT.md`](./DEVELOPMENT.md) contains the developer getting started docs -> - [`docs/`](./docs/) contains code documentation +> - [Getting Started](#getting-started) to quickly run the docker container locally +> - [`docs/`](./docs/) contains deeper code documentation > - [`frontend/`](./frontend/) contains the user interface in Svelte/TypeScript > - [`backend/`](./backend/) contains the backend HTTP server and Database -**`v0.0.0` demo / current progress** +## Getting Started - - thumbnail - +If you want to run the frontend and backend yourself, you can! Keep reading... -## Quick start +By the time you finish this guide, you will be able to run the JS frontend [Svelte](https://kit.svelte.dev/) and the Python backend [FastAPI](https://fastapi.tiangolo.com/). + +Note that we use `docker-compose` as defined in the [`docker-compose.yml`](./docker-compose.yml). + +If you want to see more specific docs go to the [ `docs/` ](./docs/README.md) for more info. + +> [!IMPORTANT] +> You must have [Docker Desktop](https://www.docker.com/products/docker-desktop/) GUI installed and the `docker-compose` bash command. + +You can run everything by doing + +```bash +sh run.sh +``` + +or directly executing + +```bash +./run.sh +``` + +this will download a docker container running the svelte frontend, the python backend, and the postgres database all in development hot-reload mode. Installation may take a few minutes. + +Now navigate to [http://localhost:5173](http://localhost:5173) to see the frontend. + +> [!TIP] +> Check the [`docs/run.md`](./docs/run.md) for documentation on the [`run.sh`](./run.sh) file for more build commands + +### Local Development Environment + +We use VSCode for all development and make sure to download a few extensions like + +- Ruff code formatter (for backend) +- Prettier code formatter (for frontend) +- Svelte (for frontend) +- Python (for backend) + +for a better experience. Then to get autocomplete with those languages, you'll need to manually install the packages that have already been installed in the docker, but locally. Otherwise your VSCode won't know how to autocomplete. + +First install the [Poetry Python Package Manager](https://python-poetry.org/) by doing + +```bash +pip3 install poetry +``` + +Then install the packages listed under [`pyproject.toml`](./backend/pyproject.toml) by doing + +```bash +poetry config virtualenvs.in-project true # required for the .venv to get created +poetry install +``` + +Then install frontend packages + +```bash +cd frontend +yarn +``` + +### Where to look + +I'd recommend going to the [`frontend/package.json`](./frontend/package.json) and [`frontend/src/App.svelte`](./frontend/src/App.svelte) if you want to get started in the frontend. + +I'd recommend going to the [`backend/src/server.py`](./backend/src/server.py) and [`backend/init.sql`](./backend/init.sql) for the backend/db. + + + +> [!TIP] +> If you have more questions, create an issue on this github or look for specific documentation in the [`docs/`](./docs/) directory -Go to [`DEVELOPMENT.md`](./DEVELOPMENT.md) to get up and running. ## Resources From f90fca257cb0365e2cac197fa1af3df3ecbc077e Mon Sep 17 00:00:00 2001 From: Donny Bertucci Date: Mon, 26 Feb 2024 21:01:06 -0800 Subject: [PATCH 7/9] Update README.md --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 568e6f91..ceea33d9 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,5 @@ -banner + +![logo-v3](https://github.com/xnought/venome/assets/65095341/9204b522-8ecf-4684-b560-4c6cd91b8e85) ![Github Actions CI tests](https://github.com/xnought/venome/actions/workflows/ci.yml/badge.svg) From ab4cd7d7b2aca2cf311a4cc75667d2ec85e5cab5 Mon Sep 17 00:00:00 2001 From: xnought Date: Mon, 26 Feb 2024 21:06:17 -0800 Subject: [PATCH 8/9] docs: include assets directly --- README.md | 2 +- docs/assets/logo-v3.svg | 24 ++++++++++++++++++++++++ 2 files changed, 25 insertions(+), 1 deletion(-) create mode 100644 docs/assets/logo-v3.svg diff --git a/README.md b/README.md index ceea33d9..222d02b4 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ +venome title -![logo-v3](https://github.com/xnought/venome/assets/65095341/9204b522-8ecf-4684-b560-4c6cd91b8e85) ![Github Actions CI tests](https://github.com/xnought/venome/actions/workflows/ci.yml/badge.svg) diff --git a/docs/assets/logo-v3.svg b/docs/assets/logo-v3.svg new file mode 100644 index 00000000..f80a063b --- /dev/null +++ b/docs/assets/logo-v3.svg @@ -0,0 +1,24 @@ + + + + + + + + + + + + + + + + + + + + + + + + From 93cef1cc93374904de1ae5bee10e669b4277bfb2 Mon Sep 17 00:00:00 2001 From: xnought Date: Mon, 26 Feb 2024 21:10:10 -0800 Subject: [PATCH 9/9] feat: better --- docs/assets/logo-v3.svg | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/docs/assets/logo-v3.svg b/docs/assets/logo-v3.svg index f80a063b..ced39745 100644 --- a/docs/assets/logo-v3.svg +++ b/docs/assets/logo-v3.svg @@ -1,20 +1,22 @@ - - - - - - + + + + + + + + - + - + - +