Skip to content
btalb edited this page Apr 15, 2021 · 10 revisions

I don't have enough space on my root drive (/) / I want to install BenchBot on a different drive.

All software installed on the host cannot easily be moved, but that does not take up much space.

The majority of BenchBot (~100GB) consists of Docker image layers. Individual docker layers can't easily be moved to a separate drive, but where Dockers stores all of its layers can be changed relatively easily. Run the following commands to stop any currently running Docker service, move all existing Docker images to a new location, add a symlink on root to the new location, & restart the Docker service:

u@pc:~$ sudo systemctl stop docker                                                      
u@pc:~$ sudo mv /var/lib/docker <NEW_LOCATION>                                         
u@pc:~$ sudo ln -sv <NEW_LOCATION> /var/lib/docker                                     
u@pc:~$ sudo systemctl start docker

(Note: the installer will offer to automatically run these commands for you if you don't have enough space at install time)

Installation fails with an error "[Errno 13] Permission denied: '...' Consider using the `--user` option or check the permissions."

Pip on your system is not currently configured to work without root permissions. The BenchBot installer does not run pip install commands as the root user, & should not need to.

Please update your system to allow use of pip without requiring root. A simple solution is to create a virtual environment that is sourced in your .bashrc:

u@pc:~$ python3 -m venv --system-site-packages myvenv
u@pc:~$ echo "source $(pwd)/myvenv/bin/activate" >> ~/.bashrc

Why can't I use Vulkan rendering with my remote setup

Please see this issue for a discussion around the causes.

In a nutshell, you can use any remote access method that still performs rendering using the hardware on your BenchBot system (i.e. the actual GPU sitting on your server). Most remote methods create virtual X sessions or pass rendering commands to your client machine, which breaks Vulkan's dependence on your server's rendering hardware.

See the following comments for extended details:

  1. https://github.com/qcr/benchbot/issues/18#issuecomment-805278637
  2. https://github.com/qcr/benchbot/issues/18#issuecomment-807727802

We welcome better solutions, but don't have the time to invest at the moment. A great alternate solution is to use OpenGL instead as described directly below.

I need to get around the Vulkan-related crashes to use BenchBot remotely

As the BenchBot Environments for Active Robotics (BEAR) are simulated using Isaac 2019.2, we follow their recommended settings and use Vulkan for rendering which, for the version of Unreal Engine being run, does not allow for headless operation.

As a workaround, we have introduced the carter_remote robot within the robots_isaac addon. This runs the simulation using OpenGL rather than Vulkan, and will operate headlessly if there's no screen to render to. This should enable remote access and operation of BenchBot via ssh.

Assuming your robots_isaac, envs_isaac_challenge, and envs_isaac_develop addons are up-to-date, headlessl operation should be as simple as the following when setting up an environment

benchbot_run --task <task_name> --env <env_name> --robot carter_remote

Warning! This approach does not use Isaac's recommended settings and it is not fully clear yet how much this will effect operation of the simulator, or the effects it may have on the tasks being performed within BenchBot.