The new message header feature in ISO specs seem to not play well with children rows

[Josephu]

Hi, I have been trying out v0.10.0, and the experience is very good. It fixed a few bugs I experienced with, and I noticed you introduced message header concept into the specs.

https://github.com/rkbalgi/isosim/blob/master/test/testdata/specs/iso_specs.yaml

However, when I tried the message header spec, it did not seem to work nicely.

Below is a workable spec.

specs:
  - name: ISO8583-UPI
    id: 1
    messages:
      - name: "Network Management"
        id: 1
        header_match:
          - "0800"
          - "0810"
          - "0820"
          - "0830"
        fields:
          - name: "Message Header"
            id: 1
            type: Fixed
            encoding: BINARY
            size: 46
            children:
              - name: "[Header] Header Length"
                id: 2
                type: Fixed
                size: 1
                data_encoding: BINARY
                position: 1
              - name: "[Header] Flag And Version"
                id: 3
                type: Fixed
                size: 1
                data_encoding: BINARY
                position: 2
              - name: "[Header] Message Length"
                id: 4
                type: Fixed
                size: 4
                data_encoding: ASCII
                position: 3
              - name: "[Header] Destination ID"
                id: 5
                type: Fixed
                size: 11
                data_encoding: ASCII
                position: 4
              - name: "[Header] Source ID"
                id: 6
                type: Fixed
                size: 11
                data_encoding: ASCII
                position: 5
              - name: "[Header] Reserved"
                id: 7
                type: Fixed
                size: 3
                data_encoding: BINARY
                position: 6
              - name: "[Header] Batch Number"
                id: 8
                type: Fixed
                size: 1
                data_encoding: BINARY
                position: 7
              - name: "[Header] Transaction Information"
                id: 9
                type: Fixed
                size: 8
                data_encoding: BINARY
                position: 8
              - name: "[Header] User Information"
                id: 10
                type: Fixed
                size: 1
                data_encoding: BINARY
                position: 9
              - name: "[Header] Reject Code"
                id: 11
                type: Fixed
                size: 5
                data_encoding: ASCII
                position: 10
          - name: "Message Type"
            id: 12
            type: Fixed
            size: 4
            data_encoding: ASCII
            constraints:
              content_type: Numeric
            children: []
          - name: "Bitmap"
            id: 13
            type: Bitmapped
            encoding: BINARY
            children:
              - name: "Transmission Date"
                id: 14
                type: Fixed
                size: 10
                data_encoding: ASCII
                position: 7
                constraints:
                  content_type: Numeric
              - name: "STAN"
                id: 15
                type: Fixed
                size: 6
                data_encoding: ASCII
                position: 11
              - name: "Forward Institution Identification Code"
                id: 16
                type: Variable
                length_indicator_size: 2
                length_indicator_encoding: ASCII
                data_encoding: ASCII
                position: 33
                constraints:
                  content_type: Numeric
                  max_size: 11
              - name: "Response Code"
                id: 17
                type: Fixed
                size: 2
                data_encoding: ASCII
                position: 39
              - name: "Network Management Information Code"
                id: 18
                type: Fixed
                size: 3
                data_encoding: ASCII
                position: 70
              - name: "Receiving Institution Identification Code"
                id: 19
                type: Variable
                length_indicator_size: 2
                length_indicator_encoding: ASCII
                data_encoding: ASCII
                position: 100
                constraints:
                  content_type: Numeric
                  max_size: 11

I tried to transform this same spec to use message header (ie. moving Message Header into header_fields), but when I try to send the same message over, it could not handle this message properly.

header_fields:
      - name: "Message Header"
        id: 1
        type: Fixed
        size: 4
        data_encoding: ASCII
        children:
              - name: "[Header] Header Length"
                id: 2
                type: Fixed
                size: 1
                data_encoding: BINARY
                position: 1
              - name: "[Header] Flag And Version"
                id: 3
                type: Fixed
                size: 1
                data_encoding: BINARY
                position: 2
              - name: "[Header] Message Length"
                id: 4
                type: Fixed
                size: 4
                data_encoding: ASCII
                position: 3
              - name: "[Header] Destination ID"
                id: 5
                type: Fixed
                size: 11
                data_encoding: ASCII
                position: 4
              - name: "[Header] Source ID"
                id: 6
                type: Fixed
                size: 11
                data_encoding: ASCII
                position: 5
              - name: "[Header] Reserved"
                id: 7
                type: Fixed
                size: 3
                data_encoding: BINARY
                position: 6
              - name: "[Header] Batch Number"
                id: 8
                type: Fixed
                size: 1
                data_encoding: BINARY
                position: 7
              - name: "[Header] Transaction Information"
                id: 9
                type: Fixed
                size: 8
                data_encoding: BINARY
                position: 8
              - name: "[Header] User Information"
                id: 10
                type: Fixed
                size: 1
                data_encoding: BINARY
                position: 9
              - name: "[Header] Reject Code"
                id: 11
                type: Fixed
                size: 5
                data_encoding: ASCII
                position: 10
)

Since I can work around it, I am ok with this issue. I am guessing this is because header_fields feature is still early in development, but I'll report it anyway for awareness.

[rkbalgi]

Hi, I will take a look at this, but a couple of questions -

1. The "Message Header" field is Fixed of size 4 but the size of it's children exceeds 4 ( perhaps a typo? its correct value = 46 as in the first snippet)
2. In your case, the header_match should be the whole 46 bytes and not just 4 bytes like "0800", "0810" etc (which probably wont help because you have variable content in your header)

I think what you are looking for is - Define a set of fields as header fields and only some of those will account for being the header? In your case, you want to parse the first 46 (header) + 4 (for message type) and only consider 4 bytes of the message type for selection ( i.e. in header_match)

Is so, I'll consider that as a enhancement :)

[Josephu]

Just to reply to your questions:

1. Yes, it is typo. The length is suppose to be 46.
2. This one is interesting, header_match sounds to me like all 46 bytes should be correct, so there probably need to be a concept of subheader field match.

In my use case, there is a message header (fixed as 46 bytes with many fix size subfield inside), which is separate to message type (4 bytes), so it's a pretty complicated case. I could probably have just hard coded the header as 46 bytes and ignore all the details to make it work. I think it's probably an uncommon need for ppl to go to this level of granuality.

[rkbalgi]

Closing this and tracking via #68