Skip to content

Setup scripts and files for GCloud GPU (and CPU) instances

Notifications You must be signed in to change notification settings

brookisme/gcloud_gpu

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

63 Commits
 
 
 
 
 
 
 
 

Repository files navigation

GCLOUD GPU

  1. TLDR; Run two lines of code and you're done (almost)
  2. Create Instance Script
  3. Instance Setup Script
  4. Almost: The rest of the setup

TLDR;

TLDR; TLDR;
# from local computer
. create_instance.sh gpu-84 4 500

# from remote instance
. instance-setup.sh
SLIGHTLY MORE DETAIL

LOCAL COMPUTER:

# example: create instance named gpu-84 with 4 GPUs 500GB of memory 
. create_instance.sh gpu-84 4 500
# example: copy setup script to remote instance named gpu-84
gcloud compute scp instance-setup.sh gpu-84:~/
# example: ssh into instance named gpu-84
gcloud compute ssh gpu-84

# --OR-- using alias (https://github.com/brookisme/gcloud_gpu/wiki/Local-Setup#gcloud_alias)
gssh gpu-84

REMOTE INSTANCE:

# example: run setup script installing TensorFlow from pip
. instance-setup.sh

# --OR-- example: run setup script installing TensorFlow later from sources (https://github.com/brookisme/gcloud_gpu/wiki/TensorFlow:-Install-from-Sources)
. instance-setup.sh gpu skip-tf

CREATE INSTANCE

Use the script create_instance.sh to create new instances.

USAGE:

. create_instance.sh \
    <NAME> \
    <COUNT> \
    [DISK_SIZE: 200] \
    [SNAPSHOT_NAME: No snapshot]

VARS:

  • NAME: (required) name of instance

  • COUNT: (required) GPUs -1,2,4 | CPUs 0

  • DISK_SIZE:

  • SNAPSHOT_NAME (see note below):

    • must first create snapshot (see above note)
    • DISK_SIZE is ignored so pass in anything as a space filler (like "skip" in example below)

EXAMPLES:

# 4 gpu, default disk size (200GB)
. create_instance.sh gpu-84 4

# 4 gpu, from snapshot
. create_instance.sh gpu-84-snapshot-name 4 skip snapshot_name

NOTE ON SNAPSHOTS: If creating instance from a snapshot you must first create the disk

gcloud compute disks create DISK_NAME --source-snapshot SNAPSHOT_NAME

INSTANCE SETUP

Use the script instance-setup.sh to setup new instances. You can find a list of what is being installed here.

Copy setup script to instance
gcloud compute scp instance-setup.sh <INSTANCE_NAME>:~/
Run setup script

NOTE: The Anaconda install is interactive. The default for add anaconda to PATH is no -- type yes!!!

You can choose to install with TensorFlow or, if you are going to install from sources, skip the tensorflow installs. You also have the option of skipping the "geo tools" listed here.

# CPU (with TF)
. instance-setup.sh cpu

# GPU (with TF)
. instance-setup.sh

# GPU (without TF)
. instance-setup.sh gpu skip-tf

# GPU (without TF or GEO-tools)
. instance-setup.sh gpu skip-tf skip-geo

# GPU (including TF but without GEO-tools)
. instance-setup.sh gpu tf skip-geo

THE REST

There are still a small handful of steps (most of which are optional).

REQUIRED:
OPTIONAL:

REMOTE INSTANCE:

LOCAL COMPUTER:

Also consider installing TensorFlow from sources.

About

Setup scripts and files for GCloud GPU (and CPU) instances

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages