[Issue]: HIP-Basics/hello_world_cuda fails to compile

[CanYing0913]

makefile is generated by

cmake -S . -B build -D GPU_RUNTIME=CUDA
cmake --build build

result of running make GPU_RUNTIME=CUDA:

$:~/Code/rocm-examples/HIP-Basic/hello_world_cuda/build$ make GPU_RUNTIME=CUDA
Consolidate compiler generated dependencies of target hip_hello_world_cuda
[ 50%] Building CUDA object CMakeFiles/hip_hello_world_cuda.dir/main.hip.o
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h(507): warning #1056-D: returning pointer to local variable

/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h(507): warning #1056-D: returning pointer to local variable

/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h: In function ‘CUDA_RESOURCE_DESC* hipResourceDesTocudaResourceDes(const HIP_RESOURCE_DESC*)’:
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h:507:8: warning: address of local variable ‘a’ returned [-Wreturn-local-addr]
  507 |     return &a;
      |        ^~
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h:481:20: note: declared here
  481 |     CUDA_RESOURCE_DESC a;
      |                    ^
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h: In function ‘hipError_t hipMemcpyToArray(hipArray_t, size_t, size_t, const void*, size_t, hipMemcpyKind)’:
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h:2071:48: warning: ‘cudaError_t cudaMemcpyToArray(cudaArray_t, size_t, size_t, const void*, size_t, cudaMemcpyKind)’ is deprecated [-Wdeprecated-declarations]
 2071 |     return hipCUDAErrorTohipError(
      |                               ~~~~             ^                                        
/usr/include/cuda_runtime_api.h:7389:46: note: declared here
 7389 | extern __CUDA_DEPRECATED __host__ cudaError_t CUDARTAPI cudaMemcpyToArray(cudaArray_t dst, size_t wOffset, size_t hOffset, const void *src, size_t count, enum cudaMemcpyKind kind);
      |                                              ^~~~~~~~~~~~~~~~~
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h: In function ‘hipError_t hipMemcpyFromArray(void*, hipArray_const_t, size_t, size_t, size_t, hipMemcpyKind)’:
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h:2078:50: warning: ‘cudaError_t cudaMemcpyFromArray(void*, cudaArray_const_t, size_t, size_t, size_t, cudaMemcpyKind)’ is deprecated [-Wdeprecated-declarations]
 2078 |     return hipCUDAErrorTohipError(cudaMemcpyFromArray(dst, srcArray, wOffset, hOffset, count,
      |                               ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  
/usr/include/cuda_runtime_api.h:7431:46: note: declared here
 7431 | extern __CUDA_DEPRECATED __host__ cudaError_t CUDARTAPI cudaMemcpyFromArray(void *dst, cudaArray_const_t src, size_t wOffset, size_t hOffset, size_t count, enum cudaMemcpyKind kind);
      |                                              ^~~~~~~~~~~~~~~~~~~
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h: In function ‘hipError_t hipCtxDetach(hipCtx_t)’:
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h:2998:41: warning: ‘CUresult cuCtxDetach(CUcontext)’ is deprecated [-Wdeprecated-declarations]
 2998 |     return hipCUResultTohipError(cuCtxDetach(ctx));
      |                              ~~~~~~~~~~~^~~~~
/usr/include/cuda.h:5478:36: note: declared here
 5478 | __CUDA_DEPRECATED CUresult CUDAAPI cuCtxDetach(CUcontext ctx);
      |                                    ^~~~~~~~~~~
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h: In function ‘hipError_t hipDeviceComputeCapability(int*, int*, hipDevice_t)’:
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h:3006:55: warning: ‘CUresult cuDeviceComputeCapability(int*, int*, CUdevice)’ is deprecated [-Wdeprecated-declarations]
 3006 |     return hipCUResultTohipError(cuDeviceComputeCapability(major, minor, device));
      |                              ~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
/usr/include/cuda.h:4195:36: note: declared here
 4195 | __CUDA_DEPRECATED CUresult CUDAAPI cuDeviceComputeCapability(int *major, int *minor, CUdevice dev);
      |                                    ^~~~~~~~~~~~~~~~~~~~~~~~~
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h: In function ‘hipError_t hipBindTexture(size_t*, textureReference*, const void*, const hipChannelFormatDesc*, size_t)’:
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h:3127:46: warning: ‘cudaError_t cudaBindTexture(size_t*, const textureReference*, const void*, const cudaChannelFormatDesc*, size_t)’ is deprecated [-Wdeprecated-declarations]
 3127 |     return hipCUDAErrorTohipError(cudaBindTexture(offset, tex, devPtr, desc, size));
      |                               ~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/cuda_runtime_api.h:8581:46: note: declared here
 8581 | extern __CUDA_DEPRECATED __host__ cudaError_t CUDARTAPI cudaBindTexture(size_t *offset, const struct textureReference *texref, const void *devPtr, const struct cudaChannelFormatDesc *desc, size_t size __dv(UINT_MAX));
      |                                              ^~~~~~~~~~~~~~~
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h: In function ‘hipError_t hipBindTexture2D(size_t*, textureReference*, const void*, const hipChannelFormatDesc*, size_t, size_t, size_t)’:
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h:3133:48: warning: ‘cudaError_t cudaBindTexture2D(size_t*, const textureReference*, const void*, const cudaChannelFormatDesc*, size_t, size_t, size_t)’ is deprecated [-Wdeprecated-declarations]
 3133 |     return hipCUDAErrorTohipError(cudaBindTexture2D(offset, tex, devPtr, desc, width, height, pitch));
      |                               ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/cuda_runtime_api.h:8640:46: note: declared here
 8640 | extern __CUDA_DEPRECATED __host__ cudaError_t CUDARTAPI cudaBindTexture2D(size_t *offset, const struct textureReference *texref, const void *devPtr, const struct cudaChannelFormatDesc *desc, size_t width, size_t height, size_t pitch);
      |                                              ^~~~~~~~~~~~~~~~~
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h: In function ‘hipError_t hipGetTextureAlignmentOffset(size_t*, const textureReference*)’:
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h:3192:60: warning: ‘cudaError_t cudaGetTextureAlignmentOffset(size_t*, const textureReference*)’ is deprecated [-Wdeprecated-declarations]
 3192 |     return hipCUDAErrorTohipError(cudaGetTextureAlignmentOffset(offset,texref));
      |                               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~
/usr/include/cuda_runtime_api.h:8773:46: note: declared here
 8773 | extern __CUDA_DEPRECATED __host__ cudaError_t CUDARTAPI cudaGetTextureAlignmentOffset(size_t *offset, const struct textureReference *texref);
      |                                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h: In function ‘hipError_t hipLaunchCooperativeKernelMultiDevice(hipLaunchParams*, int, unsigned int)’:
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h:3220:69: warning: ‘cudaError_t cudaLaunchCooperativeKernelMultiDevice(cudaLaunchParams*, unsigned int, unsigned int)’ is deprecated [-Wdeprecated-declarations]
 3220 |     return hipCUDAErrorTohipError(cudaLaunchCooperativeKernelMultiDevice(launchParamsList, numDevices, flags));
      |                               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/cuda_runtime_api.h:4198:46: note: declared here
 4198 | extern __CUDA_DEPRECATED __host__ cudaError_t CUDARTAPI cudaLaunchCooperativeKernelMultiDevice(struct cudaLaunchParams *launchParamsList, unsigned int numDevices, unsigned int flags  __dv(0));
      |                                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h: In function ‘hipError_t hipModuleLaunchCooperativeKernelMultiDevice(hipFunctionLaunchParams*, unsigned int, unsigned int)’:
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h:3227:66: warning: ‘CUresult cuLaunchCooperativeKernelMultiDevice(CUDA_LAUNCH_PARAMS*, unsigned int, unsigned int)’ is deprecated [-Wdeprecated-declarations]
 3227 |     return hipCUResultTohipError(cuLaunchCooperativeKernelMultiDevice(launchParamsList,
      |                              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~               
/usr/include/cuda.h:13724:36: note: declared here
13724 | __CUDA_DEPRECATED CUresult CUDAAPI cuLaunchCooperativeKernelMultiDevice(CUDA_LAUNCH_PARAMS *launchParamsList, unsigned int numDevices, unsigned int flags);
      |                                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h: In function ‘hipError_t hipTexRefSetAddressMode(CUtexref, int, CUaddress_mode)’:
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h:3516:52: warning: ‘CUresult cuTexRefSetAddressMode(CUtexref, int, CUaddress_mode)’ is deprecated [-Wdeprecated-declarations]
 3516 |     return hipCUResultTohipError(cuTexRefSetAddressMode(hTexRef,dim,am));
      |                              ~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
/usr/include/cuda.h:17062:36: note: declared here
17062 | __CUDA_DEPRECATED CUresult CUDAAPI cuTexRefSetAddressMode(CUtexref hTexRef, int dim, CUaddress_mode am);
      |                                    ^~~~~~~~~~~~~~~~~~~~~~
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h: In function ‘hipError_t hipTexRefSetFilterMode(CUtexref, CUfilter_mode)’:
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h:3520:51: warning: ‘CUresult cuTexRefSetFilterMode(CUtexref, CUfilter_mode)’ is deprecated [-Wdeprecated-declarations]
 3520 |     return hipCUResultTohipError(cuTexRefSetFilterMode(hTexRef,fm));
      |                              ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~
/usr/include/cuda.h:17098:36: note: declared here
17098 | __CUDA_DEPRECATED CUresult CUDAAPI cuTexRefSetFilterMode(CUtexref hTexRef, CUfilter_mode fm);
      |                                    ^~~~~~~~~~~~~~~~~~~~~
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h: In function ‘hipError_t hipTexRefSetAddress(size_t*, CUtexref, hipDeviceptr_t, size_t)’:
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h:3524:51: warning: ‘CUresult cuTexRefSetAddress_v2(size_t*, CUtexref, CUdeviceptr, size_t)’ is deprecated [-Wdeprecated-declarations]
 3524 |     return hipCUResultTohipError(cuTexRefSetAddress(ByteOffset,hTexRef,dptr,bytes));
      |                              ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/cuda.h:16926:36: note: declared here
16926 | __CUDA_DEPRECATED CUresult CUDAAPI cuTexRefSetAddress(size_t *ByteOffset, CUtexref hTexRef, CUdeviceptr dptr, size_t bytes);
      |                                    ^~~~~~~~~~~~~~~~~~~~~
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h: In function ‘hipError_t hipTexRefSetAddress2D(CUtexref, const CUDA_ARRAY_DESCRIPTOR*, hipDeviceptr_t, size_t)’:
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h:3528:53: warning: ‘CUresult cuTexRefSetAddress2D_v3(CUtexref, const CUDA_ARRAY_DESCRIPTOR*, CUdeviceptr, size_t)’ is deprecated [-Wdeprecated-declarations]
 3528 |     return hipCUResultTohipError(cuTexRefSetAddress2D(hTexRef,desc,dptr,Pitch));
      |                              ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/cuda.h:16981:36: note: declared here
16981 | __CUDA_DEPRECATED CUresult CUDAAPI cuTexRefSetAddress2D(CUtexref hTexRef, const CUDA_ARRAY_DESCRIPTOR *desc, CUdeviceptr dptr, size_t Pitch);
      |                                    ^~~~~~~~~~~~~~~~~~~~~~~
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h: In function ‘hipError_t hipTexRefSetFormat(CUtexref, CUarray_format, int)’:
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h:3532:47: warning: ‘CUresult cuTexRefSetFormat(CUtexref, CUarray_format, int)’ is deprecated [-Wdeprecated-declarations]
 3532 |     return hipCUResultTohipError(cuTexRefSetFormat(hTexRef,fmt,NumPackedComponents));
      |                              ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/cuda.h:17016:36: note: declared here
17016 | __CUDA_DEPRECATED CUresult CUDAAPI cuTexRefSetFormat(CUtexref hTexRef, CUarray_format fmt, int NumPackedComponents);
      |                                    ^~~~~~~~~~~~~~~~~
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h: In function ‘hipError_t hipTexRefSetFlags(CUtexref, unsigned int)’:
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h:3536:46: warning: ‘CUresult cuTexRefSetFlags(CUtexref, unsigned int)’ is deprecated [-Wdeprecated-declarations]
 3536 |     return hipCUResultTohipError(cuTexRefSetFlags(hTexRef,Flags));
      |                              ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~
/usr/include/cuda.h:17305:36: note: declared here
17305 | __CUDA_DEPRECATED CUresult CUDAAPI cuTexRefSetFlags(CUtexref hTexRef, unsigned int Flags);
      |                                    ^~~~~~~~~~~~~~~~
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h: In function ‘hipError_t hipTexRefSetArray(CUtexref, hipArray_t, unsigned int)’:
/opt/rocm/include/hip/nvidia_detail/nvidia_hip_runtime_api.h:3540:46: warning: ‘CUresult cuTexRefSetArray(CUtexref, CUarray, unsigned int)’ is deprecated [-Wdeprecated-declarations]
 3540 |     return hipCUResultTohipError(cuTexRefSetArray(hTexRef,(CUarray)hArray,Flags));
      |                              ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/cuda.h:16850:36: note: declared here
16850 | __CUDA_DEPRECATED CUresult CUDAAPI cuTexRefSetArray(CUtexref hTexRef, CUarray hArray, unsigned int Flags);
      |                                    ^~~~~~~~~~~~~~~~
/usr/include/c++/11/bits/std_function.h: At global scope:
/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
  435 |         function(_Functor&& __f)
      |                                                                                                                                                 ^ 
/usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
  530 |         operator=(_Functor&& __f)
      |                                                                                                                                                  ^ 
/usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
make[2]: *** [CMakeFiles/hip_hello_world_cuda.dir/build.make:76: CMakeFiles/hip_hello_world_cuda.dir/main.hip.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/hip_hello_world_cuda.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

[Snektron]

Could you tell me which versions of ROCm, CUDA, and GCC you are using? I couldn't reproduce the issue with ROCm 6.2, CUDA 12.6, and GCC 11.4.0, and the latest version of this repository. I'm starting with the CUDA 12.6 ubuntu 22.04 docker image and install ROCm by installing the hip-dev after configuring the ROCm repositories as described here.

[CanYing0913]

Thanks for the reply.
HIP version: 6.1.40093-bd86f1708
AMD clang version 17.0.0 (https://github.com/RadeonOpenCompute/llvm-project roc-6.1.2 24193 669db884972e769450470020c06a6f132a8a065b)
cuda: 12.6
gcc: 11.4
graphic card: NVIDIA 4080
I am using the latest version of this repo.
I am using WSL2, Ubuntu 22.04LTS, nvidia-smi and hipcc work there, but /opt/rocm/bin/rocminfo does not work because it seems it only recognizes AMD GPU.

[Snektron]

I'm still not seeing the issue, even with those same versions. Could you try compiling the example in a docker container, starting for example with nvidia/cuda:12.6.0-devel-ubuntu22.04? If that still fails and you tell me the exact sequence of steps you took, then it will be much easier to help you. I'm executing something along the lines of the following:

$ docker run -it --rm --gpus=all nvidia/cuda:12.6.0-devel-ubuntu22.04 bash
$ apt update
$ apt install sudo wget cmake ninja-build git
$ # The following commands are copied in verbatim from the ROCm docs, for ROCm 6.1.2
$ wget https://repo.radeon.com/rocm/rocm.gpg.key -O - | \
    gpg --dearmor | sudo tee /etc/apt/keyrings/rocm.gpg > /dev/null
$ echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/6.1.2 jammy main" \
    | sudo tee --append /etc/apt/sources.list.d/rocm.list
$ echo -e 'Package: *\nPin: release o=repo.radeon.com\nPin-Priority: 600' \
    | sudo tee /etc/apt/preferences.d/rocm-pin-600
$ apt install hip-dev rocm-core hipcc
$ git clone https://github.com/ROCm/rocm-examples
$ cd rocm-examples/HIP-Basic/hello_world
$ cmake -S . -B build -D GPU_RUNTIME=CUDA
$ cmake --build build

Since the error is originating from a C++ standard library header, I suspect that there is an issue there somehow. This could be caused by a modified or different version of libstdc++, or maybe some other version incompatibility. Its kind of annoying that it doesn't show a more complete trace of error. Anyway hence my question to start with a docker container, which should contain a pristine version of everything.

[CanYing0913]

Interesting. Using the docker image it successfully compiles, but executing it raised another error:

$ ./hello_world_hip
Hello world from host!
Hello world from device or host!
An error encountered: "named symbol not found" at /rocm-examples/HIP-Basics/hello_world/main.hip:77

But line 77 is merely a synchronization. I was thinking maybe it fails to run on my NVIDIA GPU (i.e. the second hello world is on host instead of device)?

I am following on a different installation guide here, and that one is also how I installed ROCm on my WSL2 environment.
Before trying the examples, I was using hipcc to compile some programs. However I noticed I need to manually load the c++12 libraries, for example in my Makefile:
hipcc program.hip -o run -L /usr/lib/x86_64-linux-gpu/12
The reason I use c++12 is that when I didn't link it (or link it to /usr/lib/x86_64-linux-gpu/11), they always result in ld.errors. I will try it out in this docker container later.

[zichguan-amd]

Hi @CanYing0913, unfortunately I cannot repro the issue, but it seems to be an incompatibility between gcc and nvcc. You can try the fixes here: NVlabs/instant-ngp#119 or update to ROCm 6.1.3 and newer version of cuda and nvidia driver.