Merge pull request #9 from Intermodalics/merge/upstream

Merge upstream

.github/workflows/basic-ci.yaml

@@ -4,26 +4,23 @@ on: [push, pull_request]
 jobs:
   basic_ci:
     name: Basic CI
-    runs-on: ubuntu-18.04
+    runs-on: ubuntu-22.04
     strategy:
       matrix:
-        python-version: [3.8]
+        python-version: [3.8, '3.10']
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v3
     - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v1
+      uses: actions/setup-python@v4
       with:
         python-version: ${{ matrix.python-version }}
     - name: Install Dependencies And Self
       run: |
         python -m pip install --upgrade pip setuptools wheel
-        # Workaround for https://github.com/docker/docker-py/issues/2807
-        python -m pip install six
-        pip install codecov coverage nose
-        pip install .
+        python -m pip install -e .[test] codecov pytest-cov
     - name: Run headless tests
-      uses: GabrielBB/xvfb-action@v1
+      uses: coactions/setup-xvfb@v1
       with:
-        run: nosetests -s -v --with-coverage --cover-package rocker --exclude test_nvidia_glmark2
+        run: python -m pytest -s -v --cov=rocker -m "not nvidia"
     - name: Upload coverage to Codecov
-      uses: codecov/codecov-action@v1
+      uses: codecov/codecov-action@v3

README.md

@@ -2,11 +2,53 @@
 
 A tool to run docker images with customized local support injected for things like nvidia support. And user id specific files for cleaner mounting file permissions.
 
+## Difference from docker-compose
+
+A common question about rocker is how is it different than `docker-compose`.
+`rocker` is designed to solve a similar but different problem than `docker-compose`.
+The primary goal of `rocker` is to support the use of Docker in use cases where the containers will be effected by the local environment.
+A primary example of this is setting up file permissions inside the container to match the users outside of the container so that mounted files inside the container have the same UID as the host.
+Doing this enables quickly going in and out of different containrs while leverating the same workspace on your host for testing on different platforms etc.
+This is done by dynamically generating overlays on the same core image after detecting the local conditions required.
+
+The secondary feature that `rocker` provides that docker-compose does not address is the ability to inject extra use case specific capabilities into a container before running.
+A common example is the ability to use NVIDIA drivers on a standard published image.
+`rocker` will take that standard published image and both inject the necessary drivers into the container which will match your host driver and simultaneously set the correct runtime flags.
+This is possible to do with docker-compose or straight docker.
+But the drawbacks are that you have to build and publish the combinatoric images of all possible drivers, and in addition you need to manually make sure to pass all the correct runtime arguments.
+This is especially true if you want to combine multiple possible additional features, such that the number of images starts scaling in a polynomic manner and maintenance of the number of images becomes unmanagable quickly.
+Whereas with `rocker` you can invoke your specific plugins and it will use multi-stage builds of docker images to customize the container for your specific use case, which lets you use official upstream docker images without requiring you to maintain a plethora of parallel built images.
+
+
+
 ## Know extensions
 
-Rocker supports extensions via entry points there are some built in but you can add your own. Here's a list of public repositories with extensions.
+Rocker supports extensions via entry points there are some built in but you can add your own.
+
+### Integrated Extensions
+
+There are a number of integrated extensions here's some of the highlights.
+You can get full details on the extensions from the main `rocker --help` command.
+
+- x11 -- Enable the use of X11 inside the container via the host X instance.
+- nvidia -- Enable NVIDIA graphics cards for rendering
+- cuda -- Enable NVIDIA CUDA in the container
+- user -- Create a user inside the container with the same settings as the host and run commands inside the container as that user.
+- home -- Mount the user's home directory into the container
+- pulse -- Mount pulse audio into the container
+- ssh -- Pass through ssh access to the container.
+
+As well as access to many of the docker arguments as well such as `device`, `env`, `volume`, `name`, `network`, and `privileged`.
+
+### Externally maintained extensions
+
+Here's a list of public repositories with extensions.
 
 - Off-your-rocker: https://github.com/sloretz/off-your-rocker
+- mp_rocker: https://github.com/miguelprada/mp_rocker
+- ghrocker: https://github.com/tfoote/ghrocker
+
+
 
 # Prerequisites
 
@@ -18,11 +60,12 @@ Docker installation instructions: https://docs.docker.com/install/
 
 For the NVIDIA option this has been tested on the following systems using nvidia docker2:
 
-| Ubuntu distribution  | Linux Kernel |       Nvidia drivers      |
-| -------------------- | ------------ | ------------------------- |
-|         16.04        |     4.15     | nvidia-384 (works) <br> nvidia-340 (doesn't work)  |
-|         18.04        |              |     nvidia-390 (works)    |
-|         20.04        |    5.4.0     | nvidia-driver-460 (works) |
+| Ubuntu distribution | Linux Kernel | Nvidia drivers                                    |
+| ------------------- | ------------ | ------------------------------------------------- |
+| 16.04               | 4.15         | nvidia-384 (works) <br> nvidia-340 (doesn't work) |
+| 18.04               |              | nvidia-390 (works)                                |
+| 20.04               | 5.4.0        | nvidia-driver-460 (works)                         |
+| 22.04               | 5.13.0        | nvidia-driver-470 (works)                         |
 
 Install nvidia-docker 2: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker
 
@@ -41,13 +84,12 @@ Note, that changing this setting will lead to a `Failed to initialize NVML: Unkn
 
 ## Intel integrated graphics support
 
-For intel integrated graphics support you will need to mount through a specific device
+For [Intel integrated graphics support](https://www.intel.com/content/www/us/en/develop/documentation/get-started-with-ai-linux/top/using-containers/using-containers-with-the-command-line.html) you will need to mount the `/dev/dri` directory as follows:
 
 ```
---devices /dev/dri/card0
+--devices /dev/dri
 ```
 
-
 # Installation
 
 ## Debians (Recommended)
@@ -63,7 +105,19 @@ Rocker is available via pip you can install it via pip using
 
 `pip install rocker`
 
+## Archlinux ([AUR](https://aur.archlinux.org/))
+
+Using any AUR helper, for example, with `paru`
+
+```bash
+paru -S python-rocker
+```
+
+or 
 
+```bash
+paru -S python-rocker-git
+```
 
 ## Development
 To set things up in a virtual environment for isolation is a good way. If you don't already have it install python3's venv module.
@@ -87,19 +141,18 @@ For any new terminal re activate the venv before trying to use it.
 
 ### Testing
 
-To run tests install nose and coverage in the venv
+To run tests install the 'test' extra and pytest-cov in the venv
 
     . ~/rocker_venv/bin/activate
-    pip install nose
-    pip install coverage
+    pip install -e .[test] pytest-cov
 
-Then you can run nosetests.
+Then you can run pytest.
 
-    nosetests-3.4 --with-coverage --cover-package rocker
+    python3 -m pytest --cov=rocker
 
 Notes:
 
-- Make sure to use the python3 instance of nosetest from inside the environment.
+- Make sure to use the python3 instance of pytest from inside the environment.
 - The tests include an nvidia test which assumes you're using a machine with an nvidia gpu.
 
 
@@ -142,17 +195,21 @@ On Bionic
 
 ## Volume mount
 
-For arguments with one element not colon separated.
+### For arguments with one element not colon separated
 
-`--volume` adds paths as docker volumes. The last path must be terminated with two dashes `--`.
+`--volume` adds paths as docker volumes.
+
+**The last path must be terminated with two dashes `--`**.
 
     rocker --volume ~/.vimrc ~/.bashrc -- ubuntu:18.04
 
 The above example of the volume option will be expanded via absolute paths for `docker run` as follows:
 
     --volume /home/<USERNAME>/.vimrc:/home/<USERNAME>/.vimrc --volume /home/<USERNAME>/.bashrc:/home/<USERNAME>/.bashrc  
 
-For arguments with colon separation it will process the same as `docker`'s `--volume` option, `rocker --volume` takes 3 fields.
+### For arguments with colon separation
+
+It will process the same as `docker`'s `--volume` option, `rocker --volume` takes 3 fields.
 - 1st field: the path to the file or directory on the host machine.
 - 2nd field: (optional) the path where the file or directory is mounted in the container.
    - If only the 1st field is supplied, same value as the 1st field will be populated as the 2nd field.

codecov.yml

@@ -0,0 +1,3 @@
+comment:
+  layout: "reach, diff, flags, files"
+  behavior: default

setup.cfg

@@ -1,3 +1,12 @@
 [sdist_dsc]
 epoch: 1
-debian-version: 2intermodalics
+debian-version: 1intermodalics
+
+[tool:pytest]
+markers =
+  # Tests which require a docker engine
+  docker
+  # Tests which require an NVIDIA GPU and drivers
+  nvidia
+  # Tests which require an X11 display of some kind
+  x11

setup.py

@@ -7,6 +7,7 @@
     'empy',
     'pexpect',
     'packaging',
+    'urllib3',
 ]
 
 # docker API used to be in a package called `docker-py` before the 2.0 release
@@ -31,7 +32,7 @@
 
 kwargs = {
     'name': 'rocker',
-    'version': '0.2.6',
+    'version': '0.2.12',
     'packages': ['rocker'],
     'package_dir': {'': 'src'},
     'package_data': {'rocker': ['templates/*.em']},
@@ -41,20 +42,25 @@
             'detect_docker_image_os = rocker.cli:detect_image_os',
 	    ],
         'rocker.extensions': [
+            'cuda = rocker.nvidia_extension:Cuda',
             'devices = rocker.extensions:Devices',
             'dev_helpers = rocker.extensions:DevHelpers',
             'env = rocker.extensions:Environment',
+            'expose = rocker.extensions:Expose',
             'git = rocker.git_extension:Git',
+            'group_add = rocker.extensions:GroupAdd',
             'home = rocker.extensions:HomeDir',
-            'volume = rocker.volume_extension:Volume',
+            'hostname = rocker.extensions:Hostname',
             'name = rocker.extensions:Name',
             'network = rocker.extensions:Network',
             'nvidia = rocker.nvidia_extension:Nvidia',
+            'port = rocker.extensions:Port',
             'privileged = rocker.extensions:Privileged',
             'pulse = rocker.extensions:PulseAudio',
             'ssh = rocker.ssh_extension:Ssh',
             'ssh_server = rocker.ssh_server_extension:SshServer',
             'user = rocker.extensions:User',
+            'volume = rocker.volume_extension:Volume',
             'x11 = rocker.nvidia_extension:X11',
         ]
 	},
@@ -72,6 +78,11 @@
     'python_requires': '>=3.0',
 
     'install_requires': install_requires,
+    'extras_require': {
+        'test': [
+            'pytest'
+        ]
+    },
     'url': 'https://github.com/osrf/rocker'
 }
 

src/rocker/cli.py

@@ -1,5 +1,3 @@
-#!/usr/bin/env python3
-
 # Copyright 2019 Open Source Robotics Foundation
 
 # Licensed under the Apache License, Version 2.0 (the "License");

src/rocker/core.py

@@ -1,5 +1,3 @@
-#!/usr/bin/env python3
-
 # Copyright 2019 Open Source Robotics Foundation
 
 # Licensed under the Apache License, Version 2.0 (the "License");
@@ -127,11 +125,12 @@ def get_docker_client():
         # Validate that the server is available
         docker_client.ping()
         return docker_client
-    except (docker.errors.APIError, ConnectionError) as ex:
+    except (docker.errors.DockerException, docker.errors.APIError, ConnectionError) as ex:
         raise DependencyMissing('Docker Client failed to connect to docker daemon.'
             ' Please verify that docker is installed and running.'
             ' As well as that you have permission to access the docker daemon.'
-            ' This is usually by being a member of the docker group.')
+            ' This is usually by being a member of the docker group.'
+            ' The underlying error was:\n"""\n%s\n"""\n' % ex)
 
 
 def docker_build(docker_client = None, output_callback = None, **kwargs):