Sionna GPU support in windows with WSL2

[treste91]

I am trying to correctly install Sionna in Windows through WSL2.

I have managed to successfully setup Python 3.10.12 and Tensorflow 2.15 (with GPU). I run tf.config.list_physical_devices('GPU')" and get "[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')].

However, when I run import sionna I get "No CUDA device found; using CPU as fallback.". I do not understand why tensorflow recognizes the GPU when called on its own, while sionna cannot see the GPU.

Any help is highly appreciated!

[SebastianCa]

Hi @treste91,

this error occurs when Sionna sets the Mitsuba backend (see Mitusba 3 variants for details) during the initialization. This happens in sionna.rt.__init__ and is not related to Tensorflow. Is it possible that there is an issue with your CUDA drivers in the WSL2 installation? Could be related to #319.

[treste91]

Hi @SebastianCa,

Thank you very much for your reply!

I followed the instructions in #319, but unfortunately it did not work.

Indeed the error seems to be related to Mitsuba, but I don't know how to solve this.

Any ideas on how to fix this?

----ERROR----

2024-07-19 12:45:50.483026: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-07-19 12:45:50.648609: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-07-19 12:45:50.650859: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-07-19 12:45:50.677605: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-07-19 12:45:50.759825: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-07-19 12:45:51.434590: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2024-07-19 12:45:52.300908: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:887] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-07-19 12:45:52.402266: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:887] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-07-19 12:45:52.402325: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:887] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]
No CUDA device found; using CPU as fallback.
Traceback (most recent call last):
  File "/home/stelios/.local/lib/python3.10/site-packages/mitsuba/__init__.py", line 107, in __getattribute__
    _import('mitsuba.mitsuba_' + variant + '_ext'),
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 674, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 571, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1176, in create_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
ImportError: jit_init_thread_state(): the CUDA backend hasn't been initialized. Make sure to call jit_init(JitBackend::CUDA) to properly initialize this backend.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/stelios/.local/lib/python3.10/site-packages/sionna/rt/__init__.py", line 24, in <module>
    mi.set_variant('cuda_ad_rgb')
  File "/home/stelios/.local/lib/python3.10/site-packages/mitsuba/__init__.py", line 317, in set_variant
    _import('mitsuba.ad.integrators')
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/stelios/.local/lib/python3.10/site-packages/mitsuba/python/ad/__init__.py", line 2, in <module>
    from .integrators import *
  File "/home/stelios/.local/lib/python3.10/site-packages/mitsuba/python/ad/integrators/__init__.py", line 25, in <module>
    importlib.import_module('mitsuba.ad.integrators.' + name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/stelios/.local/lib/python3.10/site-packages/mitsuba/python/ad/integrators/prb.py", line 6, in <module>
    from .common import RBIntegrator, mis_weight
  File "/home/stelios/.local/lib/python3.10/site-packages/mitsuba/python/ad/integrators/common.py", line 8, in <module>
    class ADIntegrator(mi.CppADIntegrator):
  File "/home/stelios/.local/lib/python3.10/site-packages/mitsuba/__init__.py", line 253, in __getattribute__
    result = module.__getattribute__(key)
  File "/home/stelios/.local/lib/python3.10/site-packages/mitsuba/__init__.py", line 115, in __getattribute__
    raise AttributeError(e)
AttributeError: jit_init_thread_state(): the CUDA backend hasn't been initialized. Make sure to call jit_init(JitBackend::CUDA) to properly initialize this backend.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/stelios/.local/lib/python3.10/site-packages/mitsuba/__init__.py", line 107, in __getattribute__
    _import('mitsuba.mitsuba_' + variant + '_ext'),
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 674, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 571, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1176, in create_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
ImportError: jit_init_thread_state(): the LLVM backend is inactive because the LLVM shared library ("libLLVM.so") could not be found! Set the DRJIT_LIBLLVM_PATH environment variable to specify its path.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/stelios/Telefonica/test.py", line 4, in <module>
    import sionna
  File "/home/stelios/.local/lib/python3.10/site-packages/sionna/__init__.py", line 19, in <module>
    from . import rt
  File "/home/stelios/.local/lib/python3.10/site-packages/sionna/rt/__init__.py", line 27, in <module>
    mi.set_variant('llvm_ad_rgb')
  File "/home/stelios/.local/lib/python3.10/site-packages/mitsuba/__init__.py", line 317, in set_variant
    _import('mitsuba.ad.integrators')
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/stelios/.local/lib/python3.10/site-packages/mitsuba/python/ad/__init__.py", line 2, in <module>
    from .integrators import *
  File "/home/stelios/.local/lib/python3.10/site-packages/mitsuba/python/ad/integrators/__init__.py", line 25, in <module>
    importlib.import_module('mitsuba.ad.integrators.' + name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/stelios/.local/lib/python3.10/site-packages/mitsuba/python/ad/integrators/prb.py", line 6, in <module>
    from .common import RBIntegrator, mis_weight
  File "/home/stelios/.local/lib/python3.10/site-packages/mitsuba/python/ad/integrators/common.py", line 8, in <module>
    class ADIntegrator(mi.CppADIntegrator):
  File "/home/stelios/.local/lib/python3.10/site-packages/mitsuba/__init__.py", line 253, in __getattribute__
    result = module.__getattribute__(key)
  File "/home/stelios/.local/lib/python3.10/site-packages/mitsuba/__init__.py", line 115, in __getattribute__
    raise AttributeError(e)
AttributeError: jit_init_thread_state(): the LLVM backend is inactive because the LLVM shared library ("libLLVM.so") could not be found! Set the DRJIT_LIBLLVM_PATH environment variable to specify its path.

[treste91]

After following the same steps on Ubuntu everything works. I assume the issue is with WSL2.
Any ideas are more than welcome...

[merlinND]

Hello @treste91,

Indeed, as far as I know OptiX, which we use for ray tracing, is not supported in WSL. Unfortunately that's an upstream issue that we cannot fix in Sionna nor Mitsuba. There might be unofficial workarounds, but we haven't tested them and cannot offer support.
Also discussed in: mitsuba-renderer/mitsuba3#167

It should work on Windows overall though (outside of WSL).

[treste91]

Thank you very much @merlinND !!!

I dual-booted in order to avoid any future problems...

[jmoraispk]

Hi @treste91,

If running on WSL is really a must, there seems to be a reliable workaround via manual installs here: https://forums.developer.nvidia.com/t/problem-running-optix-7-6-in-wsl/239355/8

However, like @merlinND mentioned, it should work outside of WSL (e.g. using a conda environment in the Miniforge prompt which is available after you install miniforge)