-
Notifications
You must be signed in to change notification settings - Fork 120
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
set_fancy
does not work for some arrays with certain Bigarray.kind
types.
#671
Labels
Comments
zoj613
pushed a commit
to zoj613/zarr-ml
that referenced
this issue
Jun 28, 2024
This commit implements basic functionality. - Supported codecs include Sharding Indexed, Gzip, Crc32c, Transpose and Bytes. - Supported stores include MemoryStore and FilesystemStore. - Supported data types include Float32, Float64, Complex32 and Complex64. This limitation is simply a result of Owl limited array write functionality as reported here: owlbarn/owl#671 - All the core Zarr V3 specification details have been implemented.
zoj613
pushed a commit
to zoj613/zarr-ml
that referenced
this issue
Jun 28, 2024
This commit implements basic functionality. - Supported codecs include Sharding Indexed, Gzip, Crc32c, Transpose and Bytes. - Supported stores include MemoryStore and FilesystemStore. - Supported data types include Float32, Float64, Complex32 and Complex64. This limitation is simply a result of Owl limited array write functionality as reported here: owlbarn/owl#671 - All the core Zarr V3 specification details have been implemented.
zoj613
added a commit
to zoj613/zarr-ml
that referenced
this issue
Jul 5, 2024
`Ndarray.set_fancy*` functions unfortunately don't work for array kinds other than Float32, Float64, Complex32 and Complex64. See: owlbarn/owl#671 . As a workaround we manually set each coordinate one-at-time using the basic set function which does not suffer from this bug. It is likely much slower for large Zarr chunks but necessary for usability.
zoj613
added a commit
to zoj613/zarr-ml
that referenced
this issue
Jul 5, 2024
`Ndarray.set_fancy*` functions unfortunately don't work for array kinds other than Float32, Float64, Complex32 and Complex64. See: owlbarn/owl#671 . As a workaround we manually set each coordinate one-at-time using the basic set function which does not suffer from this bug. It is likely much slower for large Zarr chunks but necessary for usability.
I was able to implement a temporary workaround to this issue by using element-wise
List.iter2
(fun coord val ->
Owl.Dense.Ndarray.Generic.set coord val) coord_list value_list I would like to add that this issue also affects the owl/src/owl/core/owl_ndarray_transpose.ml Lines 41 to 54 in 455bde1
Unfortunately, I haven't been able to find a workaround for this |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When using
Owl.Dense.Ndarray.Generic.set_fancy
with arrays of kinds notFloat32
,Float64
,Complex32
orComplex64
, the operation fails with error messageException: Failure "_ndarray_set_fancy: unsupported operation"
, even though the kind is a validBigarray.kind
type.It appears this exception is thrown by the snippet:
owl/src/owl/core/owl_slicing_fancy.ml
Lines 92 to 105 in 600c21a
I think it is very limiting to only support just 4 kinds when the set operation should work for any of the
Bigarray.kind
types.Here is a minimal example to reproduce the error:
I expected this simple operation to work but it throws a
|Exception: Failure "_ndarray_set_fancy: unsupported operation".
exception.I would love for Owl to support more kinds as im using it to write an array library that needs to support more than the 4 basic kinds listed above.
Cc @mseri @jzstark
The text was updated successfully, but these errors were encountered: