diff --git a/docs/Manga_simple.drawio.png b/docs/Manga_simple.drawio.png new file mode 100644 index 0000000..ec561a7 Binary files /dev/null and b/docs/Manga_simple.drawio.png differ diff --git a/docs/walk-through.md b/docs/walk-through.md new file mode 100644 index 0000000..de9b684 --- /dev/null +++ b/docs/walk-through.md @@ -0,0 +1,18 @@ +# Walk Through + +Allow one to search and download Mangas from from some providers (site parser) and to optionally upload them to a cloud storage service (DropBox, MegaUpload etc.) + + +Uses argparse to parse the search term to the search menu: +- given a specific site parser, we: + - search and scrape the contents using bs4 + - return the results in a consistent format so they can be consumed by the menu +- it constructs an ASCII like menu to select the manga and volume(s) of interest +- upon selection we parse the results back to the parser with the manga and volume +- parser then downloads the volumes locally using a processing pool +- upload all files downloaded to a given file hosting site + + +The interface for each manga site class (site parsers) and uploaders use the adapter pattern, this allows one to easily add a new site by ensuring the same interface is used across all. + +# Scaling diff --git a/docs/web-app.md b/docs/web-app.md new file mode 100644 index 0000000..c684056 --- /dev/null +++ b/docs/web-app.md @@ -0,0 +1,158 @@ +# Web App + +Two versions here, one without user/relational data. + +![Simple System Design](./Manga_simple.drawio.png) + +## Simple + +### Endpoints + +#### Search Mangas + +Request: + +``` +GET /v1/search?query=dragon&sources=mangaka,mangafast + +query: STR +sources: LIST[STR] +``` + +Response: + +```json +[ + { + 'title': 'dragonball', + 'latest_volume': 98, + 'source': 'mangaka', + }, + ... +] +``` + +#### Download Mangas + +Request: + +``` +GET /v1/download?manga=dragonball&volumes=1,2,78&source=mangafast + +manga: STR +volumes: LIST[INTS] +source: STR +``` + +- Response: + +Using pre-signed S3 URLS allows us to keep the S3 bucket private and time limit the URLs access + +```json +[ + { + 'title': 'dragonball', + 'volume': 98, + 's3_url': OR null, + }, + ... +] +``` + +### Database + +As we have no relational data we can just store the information in a NoSQL database. + +#### MongoDB + +One database, one collection (`mangas`) with all documents: + +```json + { + 'title': 'dragonball', + 'volume': 98, + 's3_url': OR null, + } +``` + +## Less Simple + +Storing user data and relational data is required if we want to show users their specific manga + +### (More) Endpoints + +#### Sign Up + +- Request: + +``` +POST /v1/register + +{ + 'username': 'user', + 'password': 'pass', +} +``` + +Save password hashes to DB + +- Response: + +```json +{ + "success": true, + "message": "successfully registered" +} +``` + +#### Token + +Token can be given in `Authorization` header + +``` +POST /v1/login + +{ + 'username': 'user', + 'password': 'pass', +} +``` + +- Response: + +```json +{ + 'token': +} +``` + +### Database + +#### Postgres + +If we want to associate users with their mangas + +| id | username | password | +| --- | ---------- | --------------- | +| 1 | david.ross | \ | +| 2 | james.guy | \ | + +The manga volume is an FK to manga table + +| id | volume | manga | download_url | +| --- | ------ | ----- | ------------ | +| 1 | 98 | 2 | \ | +| 2 | 89 | 1 | \ | + +| id | manga | name | +| --- | ----- | ---------- | +| 1 | 1 | dragonball | +| 2 | 2 | bleach | + +Mapping table FK to user table and FK to volume table: + +| user_id | volume_id | +| ------- | --------- | +| 1 | 1 | +| 1 | 2 | +| 2 | 2 |