update reference files for new asdf standard versions #409
Conversation
braingram
force-pushed
the
update_ref_files
branch
from
d2751d3 to
060d8c2
Compare
| !core/complex-1.0.0 (nan+infj), !core/complex-1.0.0 (nan-infj), !core/complex-1.0.0 -1.79769313486e+308j, | ||
| !core/complex-1.0.0 1.79769313486e+308j, !core/complex-1.0.0 2.22044604925e-16j, | ||
| !core/complex-1.0.0 1.11022302463e-16j, !core/complex-1.0.0 2.22507385851e-308j, | ||
| !core/complex-1.0.0 (nan+infj), !core/complex-1.0.0 (nan-infj), !core/complex-1.0.0 -1.7976931348623157e+308j, |
There was a problem hiding this comment.
The values in the existing complex.yaml are not correct for a 64 bit float (real or imaginary portion of a complex 128, yaml will treat all numbers as 64 bit so the expanded representation should be seen for all bit-depths of complex values in this file).
>> np.finfo(np.float64).max == 1.79769313486e+308
False
>> np.finfo(np.float64).max == 1.7976931348623157e+308
True
| datatype<U: !core/ndarray-1.0.0 | ||
| data: ['', "\U00010020"] | ||
| datatype: [ucs4, 2] | ||
| datatype: [ucs4, 1] |
There was a problem hiding this comment.
It is unclear to me what unicode_spp is supposed to be illustrating. My closest guess is that the character 𐀠 is U+10020 is part of plane 1 the Supplementary Multilingual Plane so perhaps this was suppored to be smp not spp. On my system this is stored in 4 byte unicode which explains the [ucs4, 1] change in this file. I'm not sure what system or version of numpy was using 8 bytes to store this character.
>> np.unicode_('𐀠').dtype.itemsize
4The generate.py file could be updated to use U2 instead of U to generate a file that uses 8 bytes:
>> np.array(['𐀠'], dtype='U2').dtype.itemsize
8At this point I suggest we hold off making this change to generate.py and instead include the updated file in this PR as it offers a slightly different example for unicode in ASDF compared to the bmp file (which has a combined character which results in [ucs4, 2]).
| @@ -155,8 +155,8 @@ def generate(version): | |||
| filename = os.path.join(outdir, name) | |||
| func(filename + ".asdf") | |||
| with asdf.open(filename + ".asdf") as af: | |||
| af.resolve_and_inline() | |||
| af.write_to(filename + ".yaml") | |||
| af.resolve_references() | |||
There was a problem hiding this comment.
This replaces the only use of resolve_and_inline I'm able to find in any package outside of asdf. See asdf-format/asdf#1690 for a description of the resolve_and_inline deprecation which includes the suggestion to update code similar to the update in this PR (replace resolve_and_inline with resolve_references and all_array_storage='inline').
braingram
force-pushed
the
update_ref_files
branch
from
7e1c3ed to
7ad3ed8
Compare
braingram
force-pushed
the
update_ref_files
branch
from
7ad3ed8 to
6882353
Compare
braingram
force-pushed
the
update_ref_files
branch
from
6882353 to
ad0ea12
Compare
Fixes a bug in the reference file generator where generated files used the latest asdf standard.
Generates and adds reference files for asdf standard versions 1.1.0 through 1.5.0 (inclusive).
I've added a few comments describing changes to the reference files (and the script to generate them).