ncplane_rgba and rgba values vs ncpixels

[joseluis]

ncplane_rgba writes an rgba array, but that array is not of ncpixels, right? Since internally an ncpixel is rbga. So there are 2 different rgba types used in the library.

[dankamongmen]

brother, don't i know it :(. it has been the source of much confusion. there are actually kinda three: rgba, rbga, and nccell/channels, which are rgb+{2 bits of A + some crap}. the rbga business arises from endianness issues.

i hadn't considered ncplane_rgba() though. if that's going to rgba, it should probably be changed to go to rbga. or, to maintain backwards compat, we should just introduce ncplane_rbga().

BUT, these are subtle and delicate matters, and i want to think about it. i thought ncpixel was actually ABGR, for instance? to be honest i've got a bit of a sunday drunk on and am not at my sharpest regarding pixel formats at the moment. i'll return to this. thanks for bringing it up.

[joseluis]

Yeah, that was the reason.

I was looking on how to proceed with updating the functions, and specifically the htole macro which depends on different c headers... I want to avoid depending on more crates. Probably to_le and to_be u32 methods are enough. I'll take a stab at it. In the worst case I see we have already the byteorder crate already in debian and fedora..

[joseluis]

Since I don't usually work with endianness, and to avoid silly mistakes, I'd like to check first with you:

// take host byte order and turn it into network (reverse on LE, no-op on BE),
// then reverse that, guaranteeing LE. htole(x) == ltohe(x).

Are all of this operations necessary? or do we just need to guarantee a LE order? i.e. to_le

[dankamongmen]

be aware that i'm hoping to get rid of this code, as it's (a) gross and (b) broken for dynamic-endian machines, when compiled under one endianness and executed on another. i just haven't figured out the best way to do so yet.

we've unfortunately got to deal with endianness at some point, since we're both treating that field as an integer sometimes, and a char* sometimes. :/

[dankamongmen]

also, what we have there can definitely be simplified using the host_to_leXX() functionality of linux's byteorder.h, though i need find freebsd equivalents to do so (i might roll our own).

[joseluis]

If the to_le method does the same as your htole macro then it should be working righ now. I don't know yet how to test it though.

[dankamongmen]

fwiw, all RGBA etc is forced on us from external sources (ffmpeg and friends), since our native format is nccell.