🚀[FEA]: install variant that doesn't require a GPU present?

[luke-conibear]

Is this a new feature, an improvement, or a change to existing functionality?

New Feature

How would you describe the priority of this feature request

Critical (currently preventing usage)

Please provide a clear description of problem you would like to solve.

Installing earth2studio installs modulus, which installs dali, which needs a GPU present (see error below).

Is it possible to have an install variant that doesn't require a GPU present?
For example, the FourCastNet NIM uses earth2studio to preprocess input data to send to a GPU server. However, the current setup also requires a GPU be present on the client side, which may not be needed e.g., to preprocess data.

$ pip install earth2studio
...
Collecting nvidia-dali-cuda120>=1.35.0 (from nvidia-modulus>=0.6.0)
  Using cached nvidia_dali_cuda120-1.44.0.tar.gz (1.6 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [57 lines of output]
      INFO:wheel-stub:Testing wheel nvidia_dali_cuda120-1.44.0-20402542-py3-none-manylinux2014_aarch64.whl against tag py3-none-manylinux2014_aarch64
      INFO:wheel-stub:Testing wheel nvidia_dali_cuda120-1.44.0-20402542-py3-none-manylinux2014_x86_64.whl against tag py3-none-manylinux2014_x86_64
        File "/private/var/folders/bg/8zzslf8s3cn8c9j6_yhryz440000gn/T/pip-build-env-1_tjec_v/overlay/lib/python3.11/site-packages/wheel_stub/wheel.py", line 235, in download_wheel
          return download_manual(wheel_directory, distribution, version, config)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/bg/8zzslf8s3cn8c9j6_yhryz440000gn/T/pip-build-env-1_tjec_v/overlay/lib/python3.11/site-packages/wheel_stub/wheel.py", line 173, in download_manual
          raise RuntimeError(f"Didn't find wheel for {distribution} {version}")
      Traceback (most recent call last):
        File "/private/var/folders/bg/8zzslf8s3cn8c9j6_yhryz440000gn/T/pip-build-env-1_tjec_v/overlay/lib/python3.11/site-packages/wheel_stub/wheel.py", line 235, in download_wheel
          return download_manual(wheel_directory, distribution, version, config)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/bg/8zzslf8s3cn8c9j6_yhryz440000gn/T/pip-build-env-1_tjec_v/overlay/lib/python3.11/site-packages/wheel_stub/wheel.py", line 173, in download_manual
          raise RuntimeError(f"Didn't find wheel for {distribution} {version}")
      RuntimeError: Didn't find wheel for nvidia-dali-cuda120 1.44.0
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/Users/lukeconibear/mambaforge/envs/dev/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/lukeconibear/mambaforge/envs/dev/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/lukeconibear/mambaforge/envs/dev/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 152, in prepare_metadata_for_build_wheel
          whl_basename = backend.build_wheel(metadata_directory, config_settings)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/bg/8zzslf8s3cn8c9j6_yhryz440000gn/T/pip-build-env-1_tjec_v/overlay/lib/python3.11/site-packages/wheel_stub/buildapi.py", line 29, in build_wheel
          return download_wheel(pathlib.Path(wheel_directory), config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/bg/8zzslf8s3cn8c9j6_yhryz440000gn/T/pip-build-env-1_tjec_v/overlay/lib/python3.11/site-packages/wheel_stub/wheel.py", line 237, in download_wheel
          report_install_failure(distribution, version, config, exception_context)
        File "/private/var/folders/bg/8zzslf8s3cn8c9j6_yhryz440000gn/T/pip-build-env-1_tjec_v/overlay/lib/python3.11/site-packages/wheel_stub/error.py", line 67, in report_install_failure
          raise InstallFailedError(
      wheel_stub.error.InstallFailedError:
      *******************************************************************************
      
      The installation of nvidia-dali-cuda120 for version 1.44.0 failed.
      
      This is a special placeholder package which downloads a real wheel package
      from https://pypi.nvidia.com/. If https://pypi.nvidia.com/ is not reachable, we
      cannot download the real wheel file to install.
      
      You might try installing this package via
      ```
      $ pip install --extra-index-url https://pypi.nvidia.com/ nvidia-dali-cuda120
      ```
      
      Here is some debug information about your platform to include in any bug
      report:
      
      Python Version: CPython 3.11.10
      Operating System: Darwin 24.2.0
      CPU Architecture: arm64
      nvidia-smi command not found. Ensure NVIDIA drivers are installed.
      
      *******************************************************************************
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[NickGeneva]

Thanks @luke-conibear

I agree, we do need to have a better minimal install option for things that dont necessarily require a GPU. Right now the best way is to install inside a docker container that has drivers inside of it but we do need to improve this.