-
Notifications
You must be signed in to change notification settings - Fork 531
Google Cloud guide (lc0)
This guide will allow you to have Leela Chess Zero clients running in the cloud in 10 minutes or less. These clients will run self-play training games and help make Leela stronger. This guide is aimed at everyone and assumes no technical understanding.
If you've never had Google Cloud account before then this is something you can do for free. (For more information about the free trial click here)!
Note, that if you're on the free trial you will NOT be charged if you run out of credits and forget to stop the trial (see 'Do I pay anything for the free trial?' here).
Create a free account on Google Cloud Platform.
Once you've created an account, navigate to Google Cloud Compute. In the interface, this is also listed as "Compute Engine."
Here you should have the option to create a VM instance. Click Create. On the Create an instance page you have many options. You can ignore most of these for now. Here are the ones you need to modify:
- Machine type: Here you can select your memory and number of CPUs. For efficiency, I recommend running a GPU on your cloud instance, you must first request a quota upgrade, since by default your free trial GPU quota is 0. Once the admins grant you additional GPUs, select a zone that contains GPUs and then follow the instructions to add GPUs. To reduce the cost of the GPU, select a pre-emptable instance. For example, selecting a Tesla K80 GPU costs only $0.22/hour
Note that you get $300 in credit free for the period of the free trial. With these settings, your virtual machine will cost about $150 per month ($165/month with 1 CPU and 1 GPU), meaning the free trial will help you train Leela for about two whole months!
- Boot disk: Select 'Ubuntu 16.04 LTS'
Now click on the 'Management, disks, networking, SSH keys' drop-down. Under 'Startup script' copy and paste the following:
#!/bin/bash
echo "Installing CUDA"
wget -nc http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1704/x86_64/cuda-repo-ubuntu1704_9.0.176-1_amd64.deb
sudo apt-get install -y --fix-missing --no-install-recommends dirmngr gnupg-curl
sudo dpkg -i cuda-repo-ubuntu1704_9.0.176-1_amd64.deb
sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1704/x86_64/7fa2af80.pub
sudo apt-get update
sudo mkdir /usr/lib/nvidia
sudo apt-get install -y --fix-missing --no-install-recommends nvidia-384=384.111-0ubuntu1 libcuda1-384=384.111-0ubuntu1 nvidia-384-dev=384.111-0ubuntu1
sudo apt-get install -y --fix-missing --no-install-recommends cuda-cudart-9-0 cuda-cublas-9-0 cuda-core-9-0 cuda-cublas-dev-9-0 cuda-cudart-dev-9-0
cd /usr/local/ && sudo ln -s cuda-9.0 cuda
export PATH="$PATH:/usr/local/cuda/bin"
cd ~
echo "Installing CUDNN"
wget -nc http://developer.download.nvidia.com/compute/redist/cudnn/v7.1.4/cudnn-9.0-linux-x64-v7.1.tgz
tar -xzvf cudnn-9.0-linux-x64-v7.1.tgz
sudo cp -P cuda/include/cudnn.h /usr/local/cuda/include/cudnn.h
sudo cp -P cuda/lib64/libcudnn* /usr/local/cuda/lib64/
sudo chmod a+r /usr/local/cuda/lib64/libcudnn*
sudo nvidia-smi -pm 1
echo "Installing other"
sudo apt-get install -y ninja-build libprotobuf-dev protobuf-compiler python3-pip
sudo pip3 install meson
sudo apt-get install -y g++-6
echo "Installing lc0"
git clone -b release --recurse-submodules https://github.com/LeelaChessZero/lc0.git
cd lc0
CC=clang-6.0 CXX=clang++-6.0 ./build.sh
mv ./build/release/lc0 lc0
echo "Downloading lczero client"
sudo curl -s -L https://github.com/LeelaChessZero/lczero-client/releases/latest | egrep -o '/LeelaChessZero/lczero-client/releases/download/.*/client_linux' | head -n 1 | wget --base=https://github.com/ -i - -O client_linux && chmod +x client_linux
echo "Running Leela Chess Zero"
./client_linux -backend-opts="cudnn(minibatch-size=512)" --user googlecloud --password googlecloud --use-test-serverAlso you may want to change the "googlecloud" username and password to your own username and password so you can see how many games you are contributing.
And that's most of the hard work done. Now grab yourself a drink while the VM boots up and the startup script runs (which should take around 3-5 minutes).
You can monitor your VMs by double clicking on your instance name on the 'VM instances' page. This will direct you to the 'VM Settings' page where you can see how much of your instances CPU is being used up (along with many other things).
After about 20-30 minutes, games from your clients will complete. After this happens, you'll be able to see the number of games you've contributed on the main page, lczero.org, or testserver.lczero.org.
Occasionally, bugs or client version updates will require the client to be restarted. You can check to see if this is needed by monitoring the main page. At the time of writing, the client is on version 16. If a new version is released, simply delete the current instance on Google Cloud and repeat the steps above.
Users who want to maximize their Google Cloud credits should use "preemptible" instances, which Google offers at an ~80% discount. The downside is that Google can turn off these instances when they please, and each instance will only run for a maximum 24 hours before being turned off.
However, restarting the preemptible instance after a shutdown is a 1-click operation from the Google management page (or one command using Google Cloud Shell) -- the disk and instance do not get deleted, just turned off. Creating an instance template with the above startup script can allow google cloud to automatically restart your pre-emptible instances for you.