leginon-tutorial

Docker container image for leginon with simulated data for tutorial purpose

release

v1.0 used in 2019 Erice workshop

Prerequisites

• A modern operating system (Linux, MacOS, Windows)

8 GB of RAM

at least 2 GB storage space for the docker container image

• A display with 1920 x 1200 resolution.

• A mouse with middle click capacity

• Docker

Set memory to 4 GB instead of the default 2 GB

• VNC Viewer: TigerVNC VNC viewer(preferred) or RealVNC.

MacOS: Do not use remote desktop included with your Mac. Too slow.

Do you have Docker installed? If not, click here

Note: You must have sudo or root access to install Docker. If you do not wish to run Docker as sudo/root, you need to configure user groups as described here: https://docs.docker.com/install/linux/linux-postinstall/

Linux    (command line)

click to expand

Download and install Docker 1.21 or greater for Linux

Consider using a Docker 'convenience script' to install (search on your OS's Docker installation webpage).

Launch docker according to your Docker engine's instructions, typically docker start.

Set memory to 4 GB from Docker Advanced Preference

MacOS 10.12 and above    (GUI)

click to expand

Download and install Docker Desktop 2.0.0 or greater for MacOS.

Set memory to 4 GB from Docker Advanced Preference

Windows    (GUI & command line)

click to expand

Download and install Docker Toolbox for [Windows](https://docs.docker.com/toolbox/toolbox_install_windows/).

Set memory to 4 GB

Launch Kitematic.

If on first startup Kitematic displays a red error suggesting that you run using VirtualBox, do so.

Do you have TigerVNC or RealVNC Viewer on your host ? If not, click here

• Download and install a VNC viewer such as TigerVNC or RealVNC:

https://tigervnc.org/

https://www.realvnc.com/en/connect/download/viewer/macos/

Installation

Do you have a Leginon-Tutotial container running in your Docker already ? If not, click here

Note: The installation directory will contain the data directory.

Linux/MacOS    (command line)

click to expand

cd
git clone http://github.com/nysbc/leginon-tutorial
cd leginon-tutorial
./run.sh

This performs the following operations (click to expand):

• Downloads the semc/leginon-tutorial repository from github saved into a local leginon-tutorial directory under your home directory.

• Build a docker image from that,

• Downloads the nysbc/leginon-tutorial repository from github saved into a local leginon-tutorial directory.

• Download the semc/leginon-tutorial image from Docker Hub to your local docker.

• Creates a Docker volume to persist the Mariadb database,

• Mounts ~/leginon-tutorial/emg/data on the host side to /emg/data inside the running container,

• Mounts the mariadb-database Docker volume to /var/lib/mysql inside the running container,

• Mounts the ~/leginon-tutorial directory to /local_data inside the running container,

• Opens ports 8000 for web traffic, 33060 for database traffic, and 5901 for VNC'ing on the host side to forward into the running container as needed.

• Waits for the mysqld_safe database daemon to launch (for ~10 seconds, but could in rare instances take longer).

Windows    (GUI & command line)

click to expand

In the Kitematic search bar type: semc/leginon-tutorial.

Click Create on the semc leginon-tutorial repository and wait for the image to download and launch.

Click for image

In Windows, you may need to tweak your settings (click to expand)

If you needed to run Docker using VirtualBox, then you will need to increase the amount of RAM allocated to Docker:

• Stop the Docker container,

• Open VirtualBox,

• Shut down the running virtual machine,

• Edit the Settings for the virtual machine you just shut down,

• Increase the RAM to 8+ GB,

• Re-start the container.

  Click for images

  

  

  • Close Kitematic

  

  

  • Increase the allocated RAM for the virtual machine:

  

  • Start Kitematic

VNC access and web access url for your docker container.

Write them down. This is how you access it later. Click to expand

Linux/MacOS

Click to expand

VNC access url: localhost:5901

• password: leginon-tutorial

Web access:

one of the two ways:

• From host: start any browser and goto this url: http://localhost:8000/myamiweb/

• Inside the docker container: start firefox and goto this url: 0.0.0.0:/myamiweb/

Windows    (GUI)

Click to expand

• To find the port, in Kitematic click on Settings the Hostname/Ports.

• The address and port next to Docker Port 5901 is for vnc access. In the example shown this means

192.168.99.100:32771 password: leginon-tutorial

Click for images

• The address and port next to Docker Port 80 is for web access. In the example shown this means

192.168.99.100:32770/myamiweb/

Click for images

• This should open up your web browser to the displayed address:

• Append /myamiweb/ to the address:

terminal access to your docker container.

Through root access of the container(click to expand)

From your host computer terminal:

cd
cd leginon_tutorial
./exec.sh
container_root>su leginonuser

Testing the installation

VNC into your leginon-tutorial container using the VNC access url found above:

password: leginon-tutorial

Click for images

• The VNC viewer should open:

• This allows you to run commands in the image xterm

From an xterm in the docker container VNC window, type:

start-leginon.py

You should see leginon setup wizard gui appears like this. You now have a working leginon-tutorial installation.

Troubleshooting

leginon gui does not start

RuntimeError: Must create at least one project before starting Leginon

Database was not initialized properly.

Reset the database and data by running this script inside the docker container.

/sw/resetdata.sh

Usage

Launch Docker

Make sure Docker is running:

• On CentOS or Ubuntu, type: sudo systemctl start docker.
  
• On OSX with Docker Desktop, Start it from Application Launch Pad.
  
• On Windows, Kitematic launches Docker automatically.

Run the container

Make sure you have the docker container running:

Linux/MacOS    (command line)

docker ps

IMAGE field should include "leginon-tutorial" if its container is running.

Windows    (gui)

Select the image in the gui and see if the gui gives you the option to stop. If it does, it is running.

To RUN and to STOP the container :

Linux/MacOS    (command line)

Click to expand

• Navigate to the leginon-tutorial installation directory to access various commands

cd
cd leginon-tutorial

• To launch a running container of the leginon-tutorial image, do:

./run.sh

• To stop the container running (this does not delete the container), do:

./kill.sh (Warning, this will stop any other Docker containers you have running as well, use with caution!)

Click for additional Docker Leginon-Tutorial container commands

• To ssh into the container as root, do:

./exec.sh

• To remove the container, but not the mounted data in /emg/data or the volume at /var/lib/mysql, do:

./rmContainers.sh (Warning: This will delete any other inactive containers you may have running as well!)

• To delete the mariadb-database volume, do:

./rmVolume.sh (Warning: This will delete the database and anything you have uploaded in Appion! The volume will be recreated the next time you do ./run.sh, but you will have an empty Appion database!)

• To re-build the leginon-tutorial image from scratch, do:

./build.sh (This will take a while!)

Windows    (gui)

Click to expand

Select the image in the gui:

• Click "play" button to start

• Click "stop" button to stop

Reset data and database

This allows you to start over cleanly

It remove the image files in /emg/data, delete all tables in your leginondb/projectdb/ap*, and then repopulate that with fresh instruments, calibrations, and settings as if you redid an installation.*

click to expand

• Access the docker image through terminal as root. This is described earlier.

./exec.sh

• Go to where the script is

cd /sw/

source ./resetdata.sh

Uninstall

click to expand

• See Docker documentation to remove the docker images

• All files are saved under leginon-tutorial directory you created during the installation, including the data. You can remove the directory and its content just like any other directory.

What can you do with this leginon/appion installation ?

click to expand

Get familiar with Leginon/Appion gui and workflow

This is fully functional simulator. It can be used to collect fake data, do ctf estimation with CTFFind4 (included) and particle picking (DogPicker and FindEM included) live, as long as you give it enough memory. You can also import new Leginon Applications and try them out. Of course there are only limited number of the reloaded images, so some operations would behave oddly.

Learn how to set up automated hole finder

The preloaded images of sq and hl should all be possible to setup to find holes automatically with MSI-T. The gold grids are the easier of the two. Give it a try.

Hint: You may want to use multiple hole template on the c-flat grid. See leginon.org documentation for more details.

How to keep myami in this docker image up to date ?

click to expand

• Access the docker image through terminal as root. This is described earlier.

./exec.sh

• Go to the git clone of myami

cd /sw/myami

• Pull git repository

git remote set-url https://emg.nysbc.org/git/myami

git pull

Switch to another myami branch

click to expand

• /sw/myami is a git clone of myami-tutorial branch. You can switch to another branch, for example myami-beta by

cd /sw/myami

git remote set-url https://emg.nysbc.org/git/myami

git checkout myami-beta

• new versions of myami branches uses pymysql. Until I build a new docker image, you need to add this with pip

pip install pymysql==0.10.0

Questions ?

Installation: email acheng@czii.org and mention leginon-tutorial

Usage: see leginon.org