Get stream position relative to object start

[ckrenslehner]

Is there a way to get a stream position which is not absolute to the stream? I want to parse multiple elements within a stream where I need a relative stream position, because otherwise the calculations are off. See the test code below.

Outer parses the total length of Inner and Inner parses two fixed fields before parsing the rest into a vec.
Now I solved this via making a custom stream which provides a relative position. Is there any other, more elegant way?

Greetings and thanks, Christian

#[test]
fn relative_stream() {
    struct RelativeStream<T> {
        inner: T,
        starting_position: Option<u64>,
    }

    impl<T: binrw::io::Seek> RelativeStream<T> {
        /// Get the relative stream position to the starting position which is saved when `new` is called
        pub fn relative_stream_position(&mut self) -> Result<u64, binrw::io::Error> {
            let current_position = self.inner.stream_position()?;

            let relative_position = current_position
                .checked_sub(
                    self.starting_position
                        .clone()
                        .ok_or(binrw::io::Error::other("No starting position"))?,
                )
                .ok_or(binrw::io::Error::other(
                    "Current position must be greater than staring position",
                ))?;

            Ok(relative_position)
        }
    }

    impl<T: binrw::io::Seek> RelativeStream<T> {
        fn new(mut inner: T) -> Self {
            // No result can be returned here so the error handling is deferred
            let starting_position = inner.stream_position().ok();

            Self {
                inner,
                starting_position,
            }
        }
    }

    impl<T: binrw::io::Read> binrw::io::Read for RelativeStream<T> {
        fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
            self.inner.read(buf)
        }
    }

    impl<T: binrw::io::Seek> binrw::io::Seek for RelativeStream<T> {
        fn seek(&mut self, pos: binrw::io::SeekFrom) -> binrw::io::Result<u64> {
            self.inner.seek(pos)
        }
    }

    #[binrw::binrw]
    #[derive(Debug)]
    struct Outer {
        len: u8,
        #[br(args(len))]
        inner: Inner,
    }

    #[binrw::binrw]
    #[derive(Debug)]
    #[br(import(len: u8))]
    #[br(stream = r, map_stream = RelativeStream::new)]
    struct Inner {
        pre_1: u8,
        pre_2: u16,

        // Uses relative stream position to ensure that not only the first element within the stream is parsed correctly
        // Relative stream position avoids hardcoding the offset caused by `pre_1` and `pre_2`
        #[br(count = (len - r.relative_stream_position()? as u8) as usize)]
        data: Vec<u8>,
    }

    #[binrw::binrw]
    #[derive(Debug)]
    struct Multi {
        #[br(parse_with = binrw::helpers::until_eof)]
        all: Vec<Outer>,
    }

    let mut cursor = binrw::io::Cursor::new(hex!("080102FFFFFFFFFFFF060304DDDDDDDD"));
    let multi = Multi::read_le(&mut cursor).unwrap();
    dbg!(multi);
}

[ismell]

I think this might solve my use case as well. I have a struct that has a header_size in the middle of the struct. I was to add an assert on my struct that verifies that the total amount of bytes read equals the defined header_size.

e.g.,

#[binrw]
#[br(assert(header_size == relative_stream_position()))]
struct Header {
    ...
    header_size: u32,
    ...
}

Unless there is already a way to do this and I missed it.

[csnover]

For this use case you should be able to use map_stream, TakeSeekExt, and until_eof:

use binrw::io::TakeSeekExt;

#[binrw::binrw]
struct Outer {
    len: u8,
    #[br(map_stream(|s| s.take_seek(len.into())))]
    inner: Inner,
}

#[binrw::binrw]
struct Inner {
    pre_1: u8,
    pre_2: u16,

    #[br(parse_with = binrw::helpers::until_eof)]
    data: Vec<u8>,
}

[ismell]

Wow that's cool. When you frame the original problem like that, it looks very different than the header_size assert problem I was trying to solve. Could you recommend an approach for asserting the header size?
Thanks!

[csnover]

I’m not sure it makes a lot of sense to assert in that way, but I guess with the abbreviated example it is hard to say for sure. Maybe you can show more of this structure and what you are trying to do?

Based on what has been stated so far, at the start of the struct, you already need to know how much to read just to get to header_size, so if you can’t get that far, it is already a failed parse. Then, either you would read header_size - <fixed size of previous fields> bytes into a collection like Vec<u8>, or you would use if(header_size > <fixed size>) on subsequent optional header fields.

I guess the main reason to assert here would be if there was some complex parsing going on with some inner field. In this case you can add temp fields with try_calc(s.stream_position()) at the start and end of the struct, and then assert the difference between those two fields matches header_size. (PosValue type also will persistently record the position of a field, and can also be used like PosValue<()> to record a position without reading any data.)

[ckrenslehner]

Thanks a lot @csnover that's perfect :-)

[ismell]

https://android.googlesource.com/platform/system/tools/mkbootimg/+/refs/heads/main/include/bootimg/bootimg.h#305 and https://android.googlesource.com/platform/system/tools/mkbootimg/+/refs/heads/main/include/bootimg/bootimg.h#275 were the headers I was parsing. The header_size isn't necessary to correctly parse the struct. I guess I could just add a assert(header_size == XXXX) to the header_size field, but I also wanted to verify that I read the entire struct and that the cursor is at the end of the struct. The PosValue<()> trick is a good workaround.

[v1gnesh]

@csnover, you see, this seems to be a more common ask (at least there's 2 of us now!).
If the reader itself is able to maintain bytes read, and I can access that info when parsing, it'll be very handy.
I currently use a counter type wrapper over the cursor, over a Read + Seek input.
It isn't perfect, as I know there will be "gaps" in the input files, and I can't therefore assert/ensure that I've parsed all known sub-structures at least.