Skip to content

30 minutes to setup dvmdostem

Jump to bottom Edit New page
Tobey Carman edited this page Oct 1, 2015 · 2 revisions

Overview

This page gives a very high level view of how to get started with dvmdostem by using a Virtual Machine provided by Vagrant.

Vagrant is a tool that helps to provide a pre-configured Virtual Machine (VM). Vagrant helps to manage the specific settings and configurations for the VM, and provides a quick way to get a repeatable, "clean", development environment with all the appropriate packages installed so that dvmdostem and associated tools will run with minimum fuss.

Note: Depending on your specific needs and experience, Vagrant may not be the best option for you. For more options see the longer tutorial: 90-minutes-to-dvmdostem

The main simulation software is written in C++. The supporting software (visualization, pre processing, post processing, and analysis) is written in a mixture of Python, R, and shell scripts.

We have desinged dvmdostem to run on *nix systems, although the build system is not robust enough to automatically configure for a given system. We have successfully run dvmdostem on Fedora 20, Ubuntu 12.04, and MacOSX 10.9.

Host Requirements

  • Depending on the specifications of your host computer, you may need to adjust the RAM and number of CPUs given to the guest by modifying the Vagrantfile.

  • On your host computer, you will need an ssh client, your ssh keys need to be setup with Github, and you will need to have an ssh-agent running with your keys added. These components allow the Vagrant VM to make connections to Github and atlas.

  • You will need an X Windows Server installed on the host. On Fedora, Ubuntu, and OSX hosts, the X Windows system should be available through the pacakge manager if it is not already installed. On Windows hosts, see the "Special Notes" section in the longer tutorial: 90-minutes-to-dvmdostem.

Getting started

Install Vagrant: https://www.vagrantup.com/downloads.html

Install VirtualBox: https://www.virtualbox.org/wiki/Downloads

Note: Make sure to get and install the Extension Pack for Virtual Box.

Next you need a Vagrantfile with instructions so that Vagrant knows how to assemble and configure the VM. For dvmdostem the Vagrantfile and an associated directory titled vagrant/ are stored in the root of the project and are kept in version control. Clone the repo to get the files:

(host)$ git clone git@github.com:<your-fork>/dvm-dos-tem.git
(host)$ cd dvm-dos-tem

Note: Cloning the repo from github, will mean that you keep a copy of dvmdostem on your host computer, even though you will likely do most of your work on the guest VM. While slightly awkward, this is not really a problem.

Note: The Vagrant VM is setup such that the project directory on the host (dvm-dos-tem) is shared with the guest. In the guest, the folder is mounted at /vagrant/. In theory it should be possible to do most of your work on the host machine (using your normal editor) and compile and run on the guest machine, but we've encountered some issues with NetCDF, and have found it easier to simply do all work inside the guest machine.

Building your VM

The Vagrantfile is essentially a Ruby script with instructions for how to download an operating system image, install it through a Virtual Machine provider (Virtual Box), and further install packages inside the Virtual Machine.

The machine specified in the Vagrantfile is roughly this:

  • Fedora 20, ~0.75GB RAM, 4 CPUs
  • Has a handful variety of software libraries installed.
  • Has clones of dvmdostem.
  • Has settings specific to our lab or working group.

Build the VM. This can take several minutes, especailly if you do not already have the base image for the VM cached locally:

(host)$ vagrant up

Note: You will want a decent internet connection for the first time you run this!! The setup and install process (provisioning) will download a base operating system from the web and will install many packages from software repositories on the web. After your machine has been provisioned the first time you will be able to bring it up without an internet connection.

The output from vagrant up is pretty informative as to what is happening, especially when paired with the Vagrantfile.

Using your VM

If the vagrant up command succeeds without any terribly alarming warnings, you should now have a virtual machine up and running. The machine is configured as a "server" in the sense that there is not a Desktop GUI environment installed. Your primary interactions with the machine will be using the command line thru an ssh connection, however it is still possible to run many graphical applications. You will interact with the the VM in more or less the same way you would interact with atlas or another remote server. You can find out more about the machine by using the vagrant programs's command line interface:

(host)$ vagrant status

To access your VM, use:

(host)$ vagrant ssh

This will attempt to connect to github with appropriate permissions, and should eventually give you a shell on your new VM:

Last login: Thu Apr  2 18:28:00 2015 from 10.0.2.2
[vagrant@localhost ~]$

If there were no errors in the vagrant up step, then you should have copies of dvmdostem and ddtv in the home directory:

[vagrant@localhost ~]$ ls
ddtv dvm-dos-tem

In addition, the following things should be setup:

  • The devel branch is checked out in the main repo
  • Remote repository points to github.com:ua-snap named upstream
  • Already installed boost, netcdf, jsoncpp, vim, gitk, git gui, qt-creator, ipython, gdal, and more
  • Conveniences added to ~/.bashrc file
  • X-11 forwarding setup so that you can use graphical tools on your VM.

Compile the model

Using Make:

[vagrant@localhost ~]$ cd dvm-dos-tem
[vagrant@localhost dvm-dos-tem (devel)]$ make

Or using Scons:

[vagrant@localhost dvm-dos-tem (devel)]$ scons

Running the model

Single site, nothing special:

[vagrant@localhost dvm-dos-tem (devel)]$ ./dvmdostem

Multi-site

[vagrant@localhost dvm-dos-tem (devel)]$ ./dvmdostem --ctrl-file config/controlfile_regn.txt

Multi-site, different loop order:

[vagrant@localhost dvm-dos-tem (devel)]$ ./dvmdostem --ctrl-file config/controlfile_regn.txt --loop-order space-major

Running, Calibration

You will need one terminal window (vagrant ssh session) for running the model itself and an additional terminal for running the calibration viewing script. With more terminals open you can simultaneously run more instances of the calibration viewing script.

In one window, start the calibration viewer:

[vagrant@localhost dvm-dos-tem (devel)]$ ./calibration/calibration-viewer.py

Now in another window, start the simulation:

[vagrant@localhost dvm-dos-tem (devel)]$ ./dvmdostem --cal-mode

The simulation should run a 100 year "warm-up" period and then stop and ask for input. The graphs in the calibration viewer should update to show the progress. In the calibration viewer, you can change the PFT that is being displayed with the (incredibly ugly) buttons on the right. Both dvmdostem and the calibration viewer have more information visible with the --help flag.

Plotting, Pre and Post Analysis

To be continued...

Turning off your Virtual Machine

When you are finished working with the VM, you can logout and then shutdown the VM:

[vagrant@localhost ~]$ exit
(host)$ vagrant halt

More options are available. See vagrant --help

Add a custom footer
Add a custom sidebar
Clone this wiki locally