Skip to content

Commit

Permalink
Auto scripts (#143)
Browse files Browse the repository at this point in the history
* Automation scripts

* CPU freq scritps
  • Loading branch information
mj-xmr committed Nov 9, 2022
1 parent 16217e0 commit c64f70c
Show file tree
Hide file tree
Showing 16 changed files with 673 additions and 22 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@ jobs:
run: util/deps-pull.sh
- name: build & install the unmanaged dependencies
run: util/deps-build.sh
- name: build & install the 3rd party tools
run: util/build-tools.sh
- name: Configure your rig
run: util/config.sh
- name: build generic
Expand Down
58 changes: 36 additions & 22 deletions docs/automation.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The following terms will be introduced:

- *Controlling computer*: The computer, which should ideally consume very little power (think of a Mini-PC or any SoC), as it should be running for most of the time - always when the AC inverter is switched on. It will be issuing commands via Ethernet cable to *Mining rigs / computers*. It will also host the `p2pool` software.
- *Mining rig / computer*: Any computer, whose sole purpose from the perspective of `SolOptXMR`, is to mine and dissipate energy overproduction, if so required. The rig is able to react on a *controlling computer's* commands, issued via Ethernet cable.
- *Node computer*: an optional part of the infrastructure, for as long as you decide to attach your `p2pool` to an **external** Monero node, that you can trust. Otherwise, this has to be a computer, that runs 24/7 and is equipped with a relatively large (between 500GB and 1TB) SSD drive to be able to host the blockchain. An HDD won't work well with Monero's blockchain unfortunately, due to many random searches, that the node needs to perform over the blockchain. I use a low powered Mini-PC for this purpose, that is connected to the grid.
- *Node computer*: an optional part of the infrastructure, for as long as you decide to attach your `p2pool` to an **external** Monero node, that you can trust. Otherwise, this has to be a computer, that runs 24/7 and is equipped with a relatively large (between 500GB and 1TB) SSD drive to be able to host the blockchain. An HDD won't work well with Monero's blockchain unfortunately, due to many random searches, that the node needs to perform over the blockchain. I use a low powered Mini-PC for this purpose, that is connected to the grid. If you go with this option, then the `p2pool` will have to be executed from there.

## System specifics

Expand All @@ -27,7 +27,7 @@ If you are confident enough to let the system schedule execution of the commands

```bash
crontab # The command that lets you edit the schedule
5 6 * * * cd /home/yoname/SolOptXMR && ./soloptxmr.py --battery-charge-ocr --np && /bin/sh /home/yoname/temp/solar/sol-cmds.sh
5 6 * * * cd /home/USR/SolOptXMR && ./soloptxmr.py --battery-charge-ocr --np && /bin/sh /home/USR/temp/solar/sol-cmds.sh
```

## Passwordless SSH access
Expand Down Expand Up @@ -82,13 +82,13 @@ USR localhost =NOPASSWD: /usr/bin/systemctl suspend, /usr/bin/systemctl poweroff
- Update freedesktop settings
Find the following entries in the `/usr/share/polkit-1/actions/org.freedesktop.login1.policy` file:

- id="org.freedesktop.login1.set-wall-message
- id="org.freedesktop.login1.suspend
- id="org.freedesktop.login1.power-off
- `id="org.freedesktop.login1.set-wall-message`
- `id="org.freedesktop.login1.suspend`
- `id="org.freedesktop.login1.power-off`

, as well as for the file `/usr/share/polkit-1/actions/org.freedesktop.systemd1.policy`:

- id="org.freedesktop.systemd1.manage-units
- `id="org.freedesktop.systemd1.manage-units`

And for each one of them and change:

Expand Down Expand Up @@ -126,7 +126,7 @@ If all went fine, you may perform the final test of being able to put a machine
```bash
# Call this to suspend a machine remotely:
ssh -n $HOST "systemctl suspend"
# Or the below one, if you know that the machine doesn't wake up properly:
# Alternatively the below one, if you know that the machine doesn't wake up properly:
ssh -n $HOST "systemctl poweroff"
```

Expand All @@ -138,11 +138,11 @@ The list also contains requirements for other tools.

```bash
# On the controlling machine:
sudo apt install at ethtool git wakeonlan libuv1-dev libzmq3-dev libsodium-dev libpgm-dev libnorm-dev libgss-dev
sudo apt install at ethtool git wakeonlan libuv1-dev libzmq3-dev libsodium-dev libpgm-dev libnorm-dev libgss-dev libcurlpp-dev

# For each mining rig:
ssh $HOST
sudo apt install at ethtool git build-essential cmake libhwloc-dev libuv1-dev libssl-dev libreadline-dev
sudo apt install at ethtool git build-essential cmake msr-tools libhwloc-dev libuv1-dev libssl-dev libreadline-dev
exit
```

Expand Down Expand Up @@ -231,26 +231,27 @@ git checkout $(git describe --tags $(git rev-list --tags --max-count=1)) # Check
./util/build-xmrig.sh # Build the mining software
```

On the controlling computer, repeat the above steps, but instead of compiling `xmrig`, compile `p2pool` instead:
On the *Controlling Computer*, repeat the above steps, but instead of compiling `xmrig`, compile `p2pool` instead:

```bash
./util/build-p2pool.sh # Build the pool software
```

If you'd like to run your own node, there's a relevant script for that too, meant to be ran from within the *Node computer*:
If you'd like to run your own node, there's a relevant script for that too, meant to be ran from within the *Node Computer*:

```bash
./util/build-monero.sh # Build the monero daemon
```

In such case, it will be expected, that you run your `p2pool` from the *Node Computer*, rather than from the *Controlling Computer*, so you have to compile the pool here instead.

### Optional optimizations of XMRig

At least for for most of Intel CPUs, it's very beneficial to enable the [MSR](https://en.wikipedia.org/wiki/Model-specific_register) module, in order to boost the hashes calculated for the same power input.
This task is however cumbersome and varying across different CPUs that the full description should not be placed in this documentation in its fullest.
However, the minimalistic setup, that only allows to use the MSR module from a `root` account would be the following:

```bash
sudo apt install msr-tools
sudo nano /etc/default/grub
```

Expand All @@ -266,8 +267,7 @@ sudo reboot
Now allow the *XMRig* to do some fine-tuning:

```bash
cd SolOptXMR/build/xmrig
sudo scripts/randomx_boost.sh
sudo build/xmrig/scripts/randomx_boost.sh
```

```bash
Expand All @@ -289,26 +289,40 @@ sudo nano /etc/rc.local && sudo chmod +x /etc/rc.local
enter there:

```bash
sleep 60; cd /home/USR/SolOptXMR && ./util/run-xmrig.sh $(nproc) &
sleep 60; cd /home/USR/SolOptXMR && ./util/run-xmrig.sh P2POOL_IP $(nproc) &
```

before the `exit 0` line of course. You may freely choose the number of threads that you want to use by replacing the `$(nproc)` with a reasonable number.
before the `exit 0` line of course.
This will connect your XMRig installation to the P2Pool passed along as the first parameter.
You may freely choose the number of threads that you want to use by replacing the `$(nproc)` with a reasonable number.
This will run the `xmrig` miner as `root`.
In case that you've either made the effort to enable the MSR module for your user, not even no effort at all to enable the MSR module even for the `root`, then the safer alternative to the above would be:

```bash
su - USR -c "sleep 60; cd /home/USR/SolOptXMR && ./util/run-xmrig.sh $(nproc)" &`
su - USR -c "sleep 60; cd /home/USR/SolOptXMR && ./util/run-xmrig.sh P2POOL_IP $(nproc)" &`
```

### Temperature control (optional)

The temperature might be monitored and controlled by throttling down the CPU frequency, by having this script ran as `root` or alternatively as a user, after allowing the user to modify the CPU frequency with: (TODO)
The temperature might be monitored and controlled by throttling down the CPU frequency, by having this script ran as `root` or alternatively as a user, after allowing the user to modify the CPU frequency (TODO).
The initial max CPU frequency may be limited to a given value in GHz, using the `util/cpu-freq.sh` script.
In order to learn your CPU's limits, please execute the `util/cpu-freq-read.sh` script first.
Assuming that the script's output is:

```
Min = 400 MHz
Max = 4.90 GHz
Curr = 2.60 GHz
```
like on my machine, to be conservative I may set the maximal frequency to 1.6 GHz in my `/etc/rc.local` and right below it the overheat watchdog with:
```bash
sleep 5; cd /home/USR/SolOptXMR && ./util/run-temperature.py --max 70 --min 50 &
sleep 1; cd /home/USR/SolOptXMR && ./util/cpu-freq.sh 1.6
sleep 5; cd /home/USR/SolOptXMR && ./util/temperature.py --max 70 --min 50 &
```

Where `--max 70` would consider the 70°C as overheat, and `--min 50` - 50°C as the target temperature while cooling down.
Where `--max 70` would consider the 70°C as overheat, and `--min 50` - 50°C as the target temperature while cooling down, before restoring the CPU frequency having been set originally.

Notice, that although this will protect your computer from overheating, it will mess up some calculations but only to the extent, that the end results will indeed be different than expected, yet not totally erratic.
IMO, it's however better to preserve your hardware, rather than receiving a few picos more for having to settle with chips being molten down.
Expand All @@ -325,10 +339,10 @@ sudo nano /etc/rc.local && sudo chmod +x /etc/rc.local
and enter:

```bash
su - USR_P2POOL -c "sleep 40; cd /home/USR/SolOptXMR && ./util/run-p2pool.sh NODE_IP WALLET_ADDR" &
su - USR_P2POOL -c "sleep 40; cd /home/USR/SolOptXMR && ./util/run-p2pool.sh WALLET_ADDR NODE_IP" &
```

where the `NODE_IP` is the connection string for the Monero node, that has to be synced and running, while the `WALLET_ADDR` is the target address for payouts.
where the `NODE_IP` is the connection string for the Monero node, that has to be synced and running, while the `WALLET_ADDR` is the target address for payouts. If you leave the NODE_IP out, `localhost` is then assumed.

### Monero daemon autostart

Expand Down
51 changes: 51 additions & 0 deletions util/build-monero.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
#!/bin/bash -e

# Uses the Monero blockchain itself to extract the raw data.
# Warning! It needs a lot of disk space to do this. Expect even ~100 GB.

NPROC=2
REPO=monero
DIR=build
TARGETS="daemon"
AUTO_INSTALL_DEPS=false

DEPS="build-essential cmake pkg-config libssl-dev libzmq3-dev libunbound-dev libsodium-dev libunwind8-dev liblzma-dev libreadline6-dev libexpat1-dev libpgm-dev libhidapi-dev libusb-1.0-0-dev libprotobuf-dev protobuf-compiler libudev-dev libboost-chrono-dev libboost-date-time-dev libboost-filesystem-dev libboost-locale-dev libboost-program-options-dev libboost-regex-dev libboost-serialization-dev libboost-system-dev libboost-thread-dev python3 ccache"

if [ ! -z $1 ]; then
NPROC=$1
fi

if [ ! -z $2 ]; then
AUTO_INSTALL_DEPS=true
fi

echo "The '$REPO' dependencies are the following:"
echo "$DEPS"
echo ""
if [ $AUTO_INSTALL_DEPS == true ]; then
q='Y'
else
echo "Would you like to install them automatically? (Y/n)"
read q
fi
if [ $q == 'n' ]; then
echo "Skipping the installation of the dependencies."
else
echo "installing dependencies of '$REPO'."
sudo apt update
sudo apt install $DEPS
fi
mkdir -p $DIR && cd $DIR
if [ ! -d $REPO ]; then
git clone --recursive https://github.com/monero-project/$REPO.git
fi
cd $REPO
git checkout $(git describe --tags $(git rev-list --tags --max-count=1)) # Checkout the latest tag (master is risky)
git submodule init && git submodule update --remote; git submodule sync && git submodule update
mkdir -p $DIR && cd $DIR
# Mix some ghetto tricks to minimize the build size:
cmake ../ -DCMAKE_BUILD_TYPE=Release -DSTRIP_TARGETS=ON -DBUILD_SHARED_LIBS=ON -DBUILD_TESTS=OFF -DARCH="default"
make -j$NPROC $TARGETS

echo "Now synchronize the blockchain with ./util/run-monero.sh"

17 changes: 17 additions & 0 deletions util/build-p2pool.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#!/bin/bash -e

VER=2.5
# https://github.com/SChernykh/p2pool/releases
REPO=p2pool
DIR=build

mkdir -p $DIR && cd $DIR
if [ ! -d $REPO ]; then
git clone --recursive https://github.com/SChernykh/$REPO.git
fi
cd $REPO
git checkout v${VER}
mkdir -p $DIR && cd $DIR
cmake ..
make -j`nproc`

13 changes: 13 additions & 0 deletions util/build-tools.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/bin/bash -e


DEPS="ethtool wakeonlan git ccache cmake msr-tools libuv1-dev libzmq3-dev libsodium-dev libpgm-dev libnorm-dev libgss-dev libcurl4-gnutls-dev libhwloc-dev libuv1-dev libssl-dev libreadline-dev"

echo "Installing the following dependencies:"
echo "$DEPS"
sudo apt install $DEPS


./util/build-p2pool.sh
./util/build-xmrig.sh
./util/build-monero.sh 2 1
25 changes: 25 additions & 0 deletions util/build-xmrig.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/bin/bash -e

VER=6.18.1
#https://github.com/xmrig/xmrig/releases
REPO=xmrig
DIR=build


DIR_THIS="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

mkdir -p $DIR && cd $DIR
if [ -d $REPO ]; then
ls
rm $REPO -fr
fi

git clone --recursive https://github.com/$REPO/$REPO.git
cd $REPO
git checkout v${VER}
git apply "$DIR_THIS/patches/xmrig-${VER}.patch" # Here be dragons!
mkdir -p $DIR && cd $DIR
cmake .. -G "CodeBlocks - Unix Makefiles"
make -j$(nproc)
#make

33 changes: 33 additions & 0 deletions util/cpu-freq-read.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#!/bin/bash

# sudo apt install cpufrequtils
# Read min/max freq:
# cpufreq-info
# Example /etc/rc.local:
# /root/bin/cpu-freq.sh 0.8
#
# exit 0

FILE_OUT_USR=/dev/shm/cpu-freq-user.txt
FILE_OUT_MIN=/dev/shm/cpu-freq-min.txt

LIMITS=$(cpufreq-info | grep limits | head -1)
MIN=$(echo $LIMITS | awk '{print $3}')
MAX=$(echo $LIMITS | awk '{print $6}')

MIN_UNIT=$(echo $LIMITS | awk '{print $4}')
MAX_UNIT=$(echo $LIMITS | awk '{print $7}')

echo "Min = $MIN $MIN_UNIT"
echo "Max = $MAX $MAX_UNIT"


FREQ_NOW_TXT=$(cpufreq-info | grep "frequency should be within" | head -1)
FREQ_NOW=$(echo $FREQ_NOW_TXT | awk '{print $(NF-1)}')
FREQ_NOW_UNIT=$(echo $FREQ_NOW_TXT | awk '{print $(NF-0)}' | cut -c -3 ) # Cut the dot

echo "Curr = $FREQ_NOW $FREQ_NOW_UNIT"

echo "${MIN} ${MIN_UNIT}" > $FILE_OUT_MIN
echo "${FREQ_NOW} ${FREQ_NOW_UNIT}" > $FILE_OUT_USR

29 changes: 29 additions & 0 deletions util/cpu-freq.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
#!/bin/bash -e

# sudo apt install cpufrequtils
# Read min/max freq:
# cpufreq-info
# Example /etc/rc.local:
# /root/bin/cpu-freq.sh 0.8
#
# exit 0
FREQ=$1
FREQ_UNITS=$2

if [ -z $1 ]; then
echo "Provide frequency in GHz"
exit
fi

if [ -z $2 ]; then
FREQ_UNITS=Ghz
fi

echo "Setting ${FREQ}${FREQ_UNITS}"

END=$(nproc)
for cpuID in $(seq 1 $END); do
echo "Setting core $((cpuID-1)) to ${FREQ}${FREQ_UNITS}"
cpufreq-set -c $((cpuID-1)) --related --max ${FREQ}${FREQ_UNITS}
done

1 change: 1 addition & 0 deletions util/patches/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
placeholder
Loading

0 comments on commit c64f70c

Please sign in to comment.