Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add cloud config for local vm deployment #93

Merged
merged 1 commit into from
Dec 12, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 17 additions & 48 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,65 +74,34 @@ set up a [local SCION development environment](https://docs.scion.org/en/latest/
and [run a local SCION topology](https://docs.scion.org/en/latest/dev/run.html).

If you run a different operating system, you can conveniently manage Ubuntu VMs with
[Multipass](https://multipass.run/install). The following commands can be used to launch a new VM, install prerequisites
[Multipass](https://multipass.run/install). The following command can be used to launch a new VM, install prerequisites
inside the VM, install the latest version of SCION, and run a local topology with services accessible from the host
machine.
machine:

```sh
# set up VM and enable direct SSH access
# if you have sufficient resources on the host, you may want to increase the VM's resources
multipass launch --disk 10G --memory 4G --cpus 2 --name scion --cloud-init - <<EOF
users:
- name: ubuntu
sudo: ALL=(ALL) NOPASSWD:ALL
ssh_authorized_keys:
- $( cat ~/.ssh/id*.pub )
EOF
multipass shell scion
multipass launch --name scion --disk 10G --memory 4G --cpus 2 --timeout 600 \
--cloud-init multipass/cloud-config.yaml
```

This will take several minutes as it builds SCION from source (hence the increased timeout).

After the launch, you can check that the network started successfully and that you see paths:

# install prerequisites
sudo apt-get update
sudo apt-get install make python3-pip ca-certificates curl gnupg

# set up Docker
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo \
"deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
"$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker $USER
exit

# download and install SCION
```sh
multipass shell scion
git clone https://github.com/scionproto/scion
cd scion
./tools/install_bazel
./tools/install_deps
./scion.sh bazel-remote
export PATH=/home/ubuntu/.local/bin/:$PATH
make build

# enable routing to local addresses
echo "net.ipv4.conf.all.route_localnet = 1" | sudo tee -a /etc/sysctl.conf
sudo sysctl --system

# optional: run local topology and check that everything works
./scion.sh topology -c topology/tiny.topo
./scion.sh run
sleep 5

sudo systemctl status scion-network.service

cd /etc/scion-rs-integration/scion/
bin/scion showpaths --sciond $(./scion.sh sciond-addr 111) 1-ff00:0:112
```

Now you can access SCION services from the host system and forward the dispatcher UNIX socket to run integration tests.
For convenience, you can use the [test_setup.sh](./test_setup.sh) script:
For convenience, you can use the [test_setup.sh](./multipass/test_setup.sh) script:

```sh
. ./test_setup.sh
chmod 0600 ./multipass/test_id_ed25519
. ./multipass/test_setup.sh
cargo test -- --ignored
```

Expand Down
138 changes: 138 additions & 0 deletions multipass/cloud-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
#cloud-config
#
# References:
# - Cloudinit boot stages: https://cloudinit.readthedocs.io/en/latest/explanation/boot.html
# - Modules in each stage: https://stackoverflow.com/a/37190866

# ----------------------------------------
# INIT STAGE
# ----------------------------------------

# Allow password-less sudo for ubuntu and add to docker group
users:
- name: ubuntu
sudo: 'ALL=(ALL) NOPASSWD:ALL'
groups: docker

ssh_authorized_keys:
- ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMYaFgA+CDCewmTdEgr14RHMXKLRkbTTIOrARMkzY0vv testing

# Write the scripts to build SCION and run the network, as well as the systemd unit files
# that will invoke them.
write_files:
- path: /etc/scion-rs-integration/build_scion.sh
owner: "root:root"
permissions: "0755"
content: |
#!/bin/bash
set -euo pipefail

export PATH="/home/ubuntu/.local/bin/:$PATH"

git clone --depth=1 https://github.com/scionproto/scion /etc/scion-rs-integration/scion
cd /etc/scion-rs-integration/scion

./tools/install_bazel
./tools/install_deps
./scion.sh bazel-remote

make build

- path: /etc/scion-rs-integration/run_network.sh
owner: "root:root"
permissions: "0755"
content: |
#!/bin/bash
set -euo pipefail

export PATH="/home/ubuntu/.local/bin/:$PATH"
EXTERNAL_ADDRESS=$(ip route get 9.9.9.9 | sed "s/.*src \([^ ]*\).*/\1/;t;d")

cd /etc/scion-rs-integration/scion

./scion.sh topology -c topology/tiny.topo
./scion.sh run

DAEMON_ADDRESS_111=$(jq -r ".\"1-ff00:0:111\"" gen/sciond_addresses.json)
sudo iptables -t nat -I PREROUTING \
-d $EXTERNAL_ADDRESS -p tcp --match multiport --dports 30000:32000 \
-j DNAT --to $DAEMON_ADDRESS_111

- path: /etc/systemd/system/scion-network.service
content: |
[Unit]
Description=Runs a local SCION network
After=cloud-final.service

[Service]
Type=oneshot
WorkingDirectory=/etc/scion-rs-integration/scion/
ExecStart=/etc/scion-rs-integration/run_network.sh
ExecStop=/etc/scion-rs-integration/scion/scion.sh stop
RemainAfterExit=yes
User=ubuntu

[Install]
WantedBy=cloud-init.target

# Add a service and timer to recreate certificates and rerun the network
- path: /etc/systemd/system/scion-restart.service
content: |
[Unit]
Description=Restarts the local SCION network

[Service]
Type=oneshot
ExecStart=/etc/scion-rs-integration/scion/scion.sh stop
ExecStart=/etc/scion-rs-integration/run_network.sh
WorkingDirectory=/etc/scion-rs-integration/scion/
User=ubuntu

- path: /etc/systemd/system/scion-restart.timer
content: |
[Unit]
Description=Timer to restart the local SCION network

[Timer]
OnCalendar=daily

[Install]
WantedBy=timers.target

# ----------------------------------------
# CONFIG STAGE
# ----------------------------------------

# Setup docker sources
apt:
sources:
docker:
source: deb [signed-by=$KEY_FILE] https://download.docker.com/linux/ubuntu $RELEASE stable
keyid: 9DC858229FC7DD38854AE2D88D81803C0EBFCD88
keyserver: https://download.docker.com/linux/ubuntu/gpg

# ----------------------------------------
# FINAL STAGE
# ----------------------------------------

# Install required packages
packages:
- ca-certificates
- containerd.io
- curl
- docker-buildx-plugin
- docker-ce
- docker-ce-cli
- docker-compose-plugin
- gnupg
- make
- python3-pip
package_upgrade: true

runcmd:
- echo "net.ipv4.conf.all.route_localnet = 1" >> /etc/sysctl.conf
- sysctl --system
- chmod "u=rwX,g=rwX,o=rwX" /etc/scion-rs-integration
- su ubuntu /etc/scion-rs-integration/build_scion.sh
- systemctl enable scion-network.service
- systemctl start --no-block scion-network.service scion-restart.timer
7 changes: 7 additions & 0 deletions multipass/test_id_ed25519
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW
QyNTUxOQAAACDGGhYAPggwnsJk3RIK9eERzFyi0ZG00yDqwETJM2NL7wAAAJDs9PkB7PT5
AQAAAAtzc2gtZWQyNTUxOQAAACDGGhYAPggwnsJk3RIK9eERzFyi0ZG00yDqwETJM2NL7w
AAAEAIEezJSN3KZg2hFur/GgEjzyPmOfrIrUs5lu9lTK/BHsYaFgA+CDCewmTdEgr14RHM
XKLRkbTTIOrARMkzY0vvAAAAB3Rlc3RpbmcBAgMEBQY=
-----END OPENSSH PRIVATE KEY-----
14 changes: 14 additions & 0 deletions multipass/test_setup.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#!/bin/bash
#
# This script assumes you have set up multipass as described in CONTRIBUTING.md
# and the SCION VM is running.
#
# Usage: . ./test_setup.sh

VM_ADDRESS=$(multipass info scion | awk '/IPv4/{print $2}')

export SCION_DISPATCHER_PATH=/tmp/dispatcher.sock
export SCION_DAEMON_ADDRESS="[$VM_ADDRESS]:30255"

mlegner marked this conversation as resolved.
Show resolved Hide resolved
rm -f $SCION_DISPATCHER_PATH
ssh -i multipass/test_id_ed25519 ubuntu@$VM_ADDRESS -fN -L $SCION_DISPATCHER_PATH:/run/shm/dispatcher/default.sock
24 changes: 0 additions & 24 deletions test_setup.sh

This file was deleted.