Templates feature

[fatgus80]

I really like the new templates feature. Some suggestions to make it even better:

• Auto generate distinct colors - Most of the data items are displayed in grey. It would be easier to read the template if a range of colors were used.

• Big endian datatypes - The current templates only have little endian data types. Big endian support would be fantastic for working with other data from other platforms.

• Array data type - Not sure if this is already supported? Maybe you could provide a generic "array" data type for situations where the built-in types are not sufficient.

[jovibor]

• Auto generate distinct colors - Most of the data items are displayed in grey. It would be easier to read the template if a range of colors were used.

Well, the main purpose of templates is that you can set whatever color you like to whatever field through the template .json file.
"clrBk" and "clrText" are exactly for this. See template files for example.

• Big endian datatypes

Will think about that.

• Array data type

You can set whatever size of the data you like through the "size" parameter. See examples.

I will update documentation to explain templates more thoroughly when they are more complete.

[fatgus80]

Well, the main purpose of templates is that you can set whatever color you like to whatever field through the template .json file.
"clrBk" and "clrText" are exactly for this. See template files for example.

I understand this. To explain better, I was thinking that the template viewer pick some nice colors if none were defined in the template. The current example templates show most fields in grey because the colors aren't defined.

Big endian datatypes
Will think about that.

Thank you for the big endian option. Please how can I create a template that contains a mixture of big endian and little endian data? Could the endianness be defined in the template itself?

[jovibor]

The current example templates show most fields in grey because the colors aren't defined.

Not exactly correct.
I tried to explain it in the documentation.
If a field doesn't have a color then a default color for the entire struct is used.
If you have a struct with 20-30 fields it would be tedious to set colors for every field, instead you just set default for all the fields. This mechanism works as well for the nested structs.

Please how can I create a template that contains a mixture of big endian and little endian data? Could the endianness be defined in the template itself?

Do you mean that a struct would contain little and big endian data simultaneously? Is it a real world scenario?

[fatgus80]

This mechanism works as well for the nested structs.

Unless the color is defined, all fields in the nested region have the same color. This makes the fields difficult to understand the fields.
For example, in the IMAGE_FILE_HEADER section of the example IMAGE_NT_HEADERS64 template, all of the fields are grey. I would suggest using an automatically generated color if the template doesn't specify one to ensure neighbouring fields are distinct.

Do you mean that a struct would contain little and big endian data simultaneously? Is it a real world scenario?

The template feature assumes the struct contains little endian fields by default. In the big endian case, it relies on the user to know that the data is big endian and select it. It would be better if this could be set in the template itself.

Yes, occasionally, there is a mixture of big and little endian data in the same structures. For example, in the ISO9660 format there is mixed use of big/little endian in neighbouring fields: https://en.wikipedia.org/wiki/ISO_9660. A basic solution would be to allow the template default mode to be set as big/little endian. An extension of this would be to allow the default mode to be overridden per field. This would make the template feature much more flexible and allow it to easily represent both big, little and mixed endian fields.

[jovibor]

An extension of this would be to allow the default mode to be overridden per field. This would make the template feature much more flexible and allow it to easily represent both big, little and mixed endian fields.

I've added endianness per field ability, will se how it goes. Check it out.

I would suggest using an automatically generated color if the template doesn't specify one to ensure neighbouring fields are distinct.

I think it's doable, just have to figure out the best user-friendly approach.

[fatgus80]

Thank you for the recent changes.

Maybe one day the templates could also support specific data types like the data interpreter? I guess this would be a much bigger change but wanted to suggest it because it would be a truly amazing feature.

[jovibor]

I would suggest using an automatically generated color if the template doesn't specify one to ensure neighbouring fields are distinct.

I've added this option to a dialog, hope it helps.

Maybe one day the templates could also support specific data types like the data interpreter? I guess this would be a much bigger change but wanted to suggest it because it would be a truly amazing feature.

It's quite possible, all the underlying code is already here. Just need some bit of inspiration.

[fatgus80]

The recent changes to the templates feature are great. Maybe you would consider some of the following suggestions for extra features:

• Arrays - Support arrays of fields or child fields. This would allow repetitive data to be more elegantly handled.

• Array element count based on another field - As above but allow the number of elements in the array to be based on another named field in the template. This is a common data pattern.

• Field offset - Allow the offset of a field (and it's children or subsequent fields) to be defined by another named field in the template. This would allow dynamic structures to be represented. This is also a common data pattern.

• Field enabled - Allow a field (and it's children, if present) to be enabled/disabled by another field in the template. This would allow optional structures to be present or removed as necessary.

I guess the final three suggestions could be complicated to implement? They would add a huge amount of flexibility to the template feature. I could suggest the ultimate solution would be if the element count, offset or enableness could be determined based on not only another named field but an expression evaluating other field(s) using something like the "C++ Mathematical Expression Toolkit Library (ExprTk)". I don't know if this would be possible but it would open a massive number of possibilities.

[jovibor]

Basically, what you suggest is a fully functional scripting language.
Originally I thought about templates as just a simple means to help in defining some static structs of data, like IMAGE_NT_HEADERS, etc...

Implementing your suggestions is of course possible, but frankly:

• I'm not sure it would be in demand outside of very small group of users, ~1%?
• Would require good amount of work
• I don't know if it could seamlessly fit into the current .json approach, so full redesigning could be required

Maybe I'm wrong, but anyways it all sounds interesting.

• Field enabled - Allow a field (and it's children, if present) to be enabled/disabled by another field in the template. This would allow optional structures to be present or removed as necessary.

Not sure I understand this.
Don't you mind showing, how should it look like/be described in terms of the current .json files?
Or maybe even suggest a PR, will be appreciated.

[fatgus80]

I would suggest a syntax like this:

`
{
"name": "field1",
"type": "int",
},

...

{
"name": "field10",
"offset": "field1.value + 20"
"size": 4
}

...

{
"name": "field20",
"enabled": "field1.value && 32"
"size": 4
}
`

In this example, I used the syntax ".value" to indicate the value of the field:

Field10's offset is the value of field1 + 20 bytes

Field20's enabled status is the value of field 1 logically AND'd with 32. If true, the field would be enabled. A field that was disabled would be invisible and treated as not present.

I don't know if this would be possible?

[jovibor]

Of course this is all possible, the only downside is that dynamic fields would significantly complicate the implementation.
Initially, templates were conceived as a simple means for describing static data structures.