-
Notifications
You must be signed in to change notification settings - Fork 108
Miscellaneous
Deployment will be run using fabric, a Python SSH task runner.
Since the Django runserver
builtin server isn't actually a very good web server (it's designed
for one user, not hundreds), we use a different web server called gunicorn on the OCF.
Our deployment will be to the OCF's apphost server, https://apphost.ocf.berkeley.edu/. The OCF's documentation for running apps can be found here.
We download a copy of the code, from the Github master branch. This will allow us to control which version of code to deploy, and restrict push access to core developers.
We should download a copy, and not simply git pull
. This is because we want to allow rollbacks, and we don't
want rogue files laying around forever, and we want a controlled environment as much as possible.
The fake README describes the deploy process in detail, which follows the Capistrano model from Ruby:
- Inside a deploy_path, create a releases, shared, and repo directory.
- Clone a repository into the repo directory.
- Extract a configurable branch into a subdirectory of the releases directory.
- Symlink that subdirectory to a current folder in the deploy_path
- Symlink shared files/folders in shared into current.
What this means is that the folder structure in the deploy server (apphost.ocf.berkeley.edu) should look like this:
~/hknweb/
prod/
current/ -> <release symlink>
releases/
shared/
repo/
test/
current/ -> <release symlink>
releases/
shared/
repo/
<release symlink>
means that a website code timestamped folder is symlinked (similar to your conventional "shortcuts" or "alias") located inside releases/
Our website runs using gunicorn, and is managed by systemd (see the Ubuntu docs). When we deploy a new version of the website, we have to restart it using systemd:
systemctl --user restart hknweb.service
This also means that when the OCF takes down their servers for maintenance, when they are turned on our website is also turned on automatically.
This depends on two files, a run
script in our repo and a systemd unit script on the apphost: ~/.config/systemd/user/hknweb.service
.
-
hknweb.service
describes how to run and restart our server -
run
describes the actualgunicorn
command to run an instance of our server
systemd
, short for 'system daemon', is the system and service manager on Debian. It is the first program loaded on startup, and it runs all other programs. Notably, it runs journald (logs), logind (login), networkd (networking), the shell (text-based, like bash, or graphical shells like GNOME / KDE / XFCE), and other background processes (services).
Most systemd
control happens through systemctl
, most often enabling / disabling / starting / stopping services:
sudo systemctl enable mysqld
The logs (debugging output) can be accessed through journalctl
:
journalctl -e
The website is run as a systemd service unit, so that it is started on startup and automatically rebooted on crashes.
For more information, see the Debian handbook on Unix services.
Gunicorn has a green-thread worker model. This means that it controls multiple worker processes to handle requests, from a single master process which distributes requests.
Green threads, or m-to-n threads, means that every process has its own thread scheduler, instead of using the OS (kernel) thread scheduler. The idea is that since we know what kind of task we need to do, I/O heavy web requests (which the OS can't assume), we can figure out how much CPU% to give to each task ourselves, and do so more efficiently.
More technically, it means that we map any m task threads to some n OS threads.
Credits for the videos and slides go to Eric Paulos' Fall 2020 CS 160
Access CS 160 Lectures 2 and 7 here: https://drive.google.com/drive/folders/1F7Hp89Kc48oAi7vOcr8JPZPJ-GMJ2wkm
- Lecture 2: Start to 46:16, skip “Announcements” at the beginning
- Slides link: http://teaching.paulos.net/cs160_FL2020/lectures/cs160-fl20-02.pdf
- Lecture 7: 22:30 to end, but ignore User Testing
- Slides link: http://teaching.paulos.net/cs160_FL2020/lectures/cs160-fl20-07.pdf
- Wikipedia Article Link: https://en.wikipedia.org/wiki/Rubber_duck_debugging
- Purpose: Force yourself to explain your code and why it works
- You don’t need a Rubber Duck
- Any inanimate object
-
Remember: Code is just a formalized language of Logic!
-
Move away from code, make sure the Logic is right first!
-
Use Paper Brainstorming (inspiration from writers)
- short-term memory - process that involves remembering bits of data for just a few seconds
- More here: https://bebrainfit.com/short-term-memory-loss/
- Inspiration from History:
- We only remember the history written on paper
- Oral history is a game of telephone (e.g. lost languages)
- Helps with future officer cores iterations to know what has been done
- Helps you to know what needs to be done in English first before formal (Django) code
-
You are not alone!!!!!!
-
There are other people to help you and talk to
-
Do not be shy to reach out!
- PlantUML
- Can make a link to different PlantUML cells to create flow charts / database relation diagrams
- Link here to a Coursey Survey PlantUML (example only, potentially outdated)
In the past we used our own server infrastructure, but currently the OCF hosts our website(s). In particular, we manage three websites:
- https://hkn.eecs.berkeley.edu (the HKN website)
- https://prot-hkn.eecs.berkeley.edu (the HKN prot wiki)
- https://hivemind.eecs.berkeley.edu (an EECS lab computer status page)
We also have control of two domains for testing:
- https://hkn-eecs-berkeley-edu.dev-apphost.ocf.berkeley.edu (the Rails website)
- https://dev-hkn-eecs-berkeley-edu.dev-apphost.ocf.berkeley.edu (the Django website)
The OCF has excellent staff documentation for their operations. Historically, some compserv officers have served as OCF staff, so we have close relations with the OCF.
The OCF handles server upkeep for our website. They install security updates, upgrade hardware, run backups (we make database backups as well), and keep the servers running, so we only have to make sure our website keeps running.
The OCF runs on Debian Linux, a distribution of Linux focused on stability (never crashing). It is developed by over a thousand people distributed across the world. Several other derivative distributions depend on its packages, notably Ubuntu and Kali Linux (for security researchers).