Group dtypes together in a 'Struct' object

[scott-griffiths]

This is possibly mostly an internal efficiency idea, rather than anything that will be part of the public interface.

Basically I want to have a Struct class (in practice probably _Struct) that represents a sequence of known-length dtypes. It would thus have a known length itself, together with a list of dtypes that comprise it.

There could be a few advantages to this. Let's say that we want to do:

vals = s.unpack('bool*4, u12')

Right now this involves parsing the string, scanning for stretchy tokens, creating five dtypes, reading them and returning their values. We don't gain much by using a list of dtypes instead:

dl = [Dtype('bool'), Dtype('bool'), Dtype('bool'), Dtype('bool'), Dtype('u12')]
vals = s.unpack(dl)

but possibly we could more efficiently do

st = Struct('bool*4, u12')
vals = s.unpack(st)

as the unpack would know the structure up-front.

I suspect that most use-cases are with dtypes that could be converted into structs. A method like unpack could first try to create a Struct, and if it failed revert to the current code.

There are also possibilities for structs to be lazy, though perhaps not in the current API. Basically it doesn't need to parse the dtypes until they are needed - this would be super useful for the bitformat module.

So could it contain arrays as well?

sa = Struct([Array('bool', 4), Dtype('u12')])

Seems reasonable.

>>> len(st)
5  # number of dtypes
>>> len(sa)
2  # one Array, one Dtype
>>> assert st.bitlength == sa.bitlength == 16
>>> st.parse('0b1011, u12=57')
[True, False, True, True, 57]
>>> st.build([...])
...