Using __local memory inside devicelib

[pvelesko]

Any suggestions on how to best allocate and pass __local memory to functions inside devicelib kernels?

Normally, this memory would be allocated inside kernel.

@linehill @pjaaskel

[linehill]

Do you mean you want to do something like this in non-kernel function?

void foo(...) {
  static local float lcl_mem[123]; // Note: invalid OpenCL C.
  ...
}

[pvelesko]

Yes, I would like to use local memory inside device function but the issue is that I can't allocate it inside device function. Was wondering what would be the best way to approach this Sent from Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: Henry Linjamäki ***@***.***> Sent: Friday, August 2, 2024 2:46:50 PM To: CHIP-SPV/chipStar ***@***.***> Cc: Paulius Velesko ***@***.***>; Author ***@***.***> Subject: Re: [CHIP-SPV/chipStar] Using __local memory inside devicelib (Discussion #902) Do you mean you want to do something like this in non-kernel function? void foo(...) { static local float lcl_mem[123]; // Note: invalid OpenCL C. ... } — Reply to this email directly, view it on GitHub<#902 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACCJBQM6DUTCS3PNZVXUTILZPNWSVAVCNFSM6AAAAABL4HROACVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAMRSGI4TMNY>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[pvelesko]

From @pjaaskel :

| Can't you just move that local array inside the synchtreads_count() implementation?

No, because you can only allocate memory in kernel regions - so this memory needs to be allocated somewhere else and passed in as argument

[pjaaskel]

In practice, a magic function call that asks for dynamic (local size dependent) dize local memory and is expanded by the compiler to allocate it. Not sure can it be made work at SPIR-V level in general as I think automatic locals must be compile-time sized.

[pvelesko]

@pjaaskel

SPIR-V level in general as I think automatic locals must be compile-time sized.

I've found that automatic __local arrays get translated to spirv as just pointers to workgroup memory without any indication of size.

__local int var[10]

I think I found a solution that works for implementing a single device function that requires __local memory (i.e. it's not magic)

1. Declare __local memory as extern inside the device function that needs it
2. Compile device_function.cl to SPIR-V
3. Edit human readable SPIR-V and delete this line OpDecorate %localMem LinkageAttributes "localMem" Import
4. For some reason in step 2 __local gets compiled to CrossWorkgroup (__global) so replace relevant instances of CrossWorkgroup with Workgroup (__local)
5. Assemble back into machine SPIR-V.
6. Use the rtdevlib mechanism to insert this into the runtime.

This compiles and seems to be giving me a reasonable answer so far (early tests).

Given this workaround, it seems that allocating __local memory in device functions should be possible?

[pjaaskel]

I wouldn't draw that conclusion just yet. These type of tricks often work on luck on some systems and fail miserably on others. If there is no indication of size, who will allocate that local memory and ensure you won't overwrite someone else's local memory areas (likely another work-group's) etc.?

[pvelesko]

If there is no indication of size, who will allocate that local memory and ensure you won't overwrite someone else's local memory areas (likely another work-group's) etc.?

I don't understand why there is no indication of size being generated by SPIR-V.

[linehill]

| I don't understand why there is no indication of size being generated by SPIR-V.

There should be like in this example where the size of the var is encoded in %_arr_uint_ulong_10 = OpTypeArray ....

[pvelesko]

Yeah pekka messaged me yesterday and i got it cleared up. Sent from Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: Henry Linjamäki ***@***.***> Sent: Thursday, August 8, 2024 9:20:56 AM To: CHIP-SPV/chipStar ***@***.***> Cc: Paulius Velesko ***@***.***>; Author ***@***.***> Subject: Re: [CHIP-SPV/chipStar] Using __local memory inside devicelib (Discussion #902) | I don't understand why there is no indication of size being generated by SPIR-V. There should be like in this example<https://godbolt.org/z/x89dMqEn9> where the size of the var is encoded in %_arr_uint_ulong_10 = OpTypeArray .... — Reply to this email directly, view it on GitHub<#902 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACCJBQJDZZYDE7NSXDXXYQDZQME4RAVCNFSM6AAAAABL4HROACVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAMRXGE3DAMY>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[pjaaskel]

KhronosGroup/OpenCL-Docs#1221