Opioner-backend-docker-app

Goal - MAKE YOUR OPINION COUNT AND SHARE WITH WORLD

This project is meant to maintain the backend code for the Opioner - MAKE YOUR OPINION COUNT AND SHARE WITH WORLD.

Currently, the project is deployed using Render servers and all requests are served through Render API gateway.
Earlier, This was done through Heroku.
Frontend of Opioner app is served through Render pages but is planned to deploy through AWS in future.

Running application locally:

• Clone this repo
  git clone git@github.com:souravs17031999/opioner-backend.git
• Start docker service if not started already on your system (Install docker from here and docker-compose from here)
• Run all services with docker-compose from cloned project root directory
  docker-compose up --build
• Your services should be up and running.

or

• Run make local

Test the project locally:

• Run make build
• Run make test

Test the deployed project:

• Run make test_postdeploy (useful in CI/CD pipelines)

ENV VARIABLES FOR RUNNING (before starting the service):

• PGHOST
• PGUSER
• PGPASSWORD
• PGDATABASE
• REQUIRE_DB_INSERT
• POSTGRES_PASSWORD
• POSTGRES_DB
• POSTGRES_USER
• ALLOWED_ORIGIN_HOST_PROD
• REQUIRE_DB_MIGRATIONS
• REQUIRE_DB_INSERT
• SENDGRID_API_KEY_PROD
• REDIS_URL
• NOTIFICATION_INTERNAL_URL
• HEROKU_API_KEY
• SECRET_KEY
• OIDC_CONFIG

Microservices:

• Auth-service
• Product-service
• User-service
• Notification-service
• Cron-service
• Postgres
• Redis

Deployment via CI/CD:

@Live

• Deployment pipeline follows the deployment via Render CI/CD docker container registries to deliver the end product.
• Tests are integrated in the pipeline (preferably Jenkins pipeline) before and after the delivery of end product to make sure we deliver the value of the product without breaking existing functionality.

@Deprecated

• Deployment pipeline follows the deployment via Heroku CI/CD docker container registries to deliver the end product.
• Run make heroku_deploy to deploy to Heroku cluster.

Github CI pipeline:

• Every push to master branch triggers github Makefile CI actions workflow which builds, test and deploy container images.
• Additional code quality checks via SONAR quality checks have been integrated.

@Deprecated

New Relic Monitoring

• We already have integrated new relic monitoring dashboards but configuration file newrelic.ini will have to be provided (inserted at each service root dir) and set APM_MONITORING_NEWRELIC to "True" to start application service with monitoring enabled. (for security reasons, configuration file will not be commited to this repo)

Keycloak based authentication and authorization

• All routes are protected with JWT based token authentication issued by Keycloak.
• Authn and Authz are decoupled from the business logic of the microservices and terminated at the entrypoints of the microservices.
• Various types of checks such as token signature, expiration, roles etc are checked before request is sent forward to actual view mapped for the route.

Pagination in API's

• Some API's such as for fetching public feeds which contains long lists of feeds data generated by various users is paginated for better client side experience so as to making rendering smooth, limit the time on API response (database query) and overall avoiding network timeouts/delays.

Databases (volume persistence)

@Live

• This project now uses Supabase managed Postgres Databases.
  @Deprecated
• Currently, I have explored and made available instances of Heroku postgresDB and Amazon RDS postgresDB in production.
• For developmental debugging, volumes persistence (bind mounts/named volumes) is included in docker-compose.yml (using docker-compose)

@deprecated

Firebase storage

• Firebase is used for storage of profile pictures.

Useful:

• Each service is built independently using it's own Dockerfile and pushed to Heroku container registeries.
• Then, the container is released into production.
• Final built image (for every service) contains the ENTRYPOINT which starts the flask api service and runs any scripts supplied before actually starting the service.
• Every service waits for the postgres service to start first (controlled by script), as every service depends on this one.
• Repo contains db migration scripts for local setup.
• Migration scripts take care of creating initial schema on local setup (from postgres DB docker) and production setup (from auth service migrations phase).

Docker container optimization

• We have developed base-python-image which contains all the dependencies pre-installed specific to opioner application requirements that are common for consistency among all microservices and improves build time execution. python-base-image

Notes:

• The actual database for app will be mounted automatically at dbdata directory owned by docker.
• You can use Jenkins build locally for testing your changes as it already contains test stage which needs to pass all the tests added.
• For testing purpose, it will automatically create the test database in Jenkins build and will be removed after the container stops.
• Jenkins CI/CD is already configured in Jenkinsfile and Makefile.
• We will use dockers to deploy the app in production.
• dbinit contains test data to be migrated while starting postgres container and can be used for testing.
• Currently, db migration step has been added in auth-service, in future it maybe be segragated.
• Currently, we have total 5 services: auth-service, user-service, product-service, notification-service, cron-service.
• All environment variables in Dockerfile are just default values, actual values are injected on the fly while deployment (not commited due to security reasons).

Rest API's have been designed using best practices which conveys the exact purpose of what each API does

opioner-commons contains the useful scripts to be run on Jenkins CI/CD pipeline

Hot reloading: enabled for all services, server is reloaded as soon as code change is detected in development

GDPR compliance:

• All the data processed for user is transparent and is explained on terms and privacy policy page of (Opioner-home website)[https://souravs17031999.github.io/opioner-home]

Whats coming up ? Delpoyment strategy via Terraform and Kubernetes soon coming up.