Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Revamp unpack to use position #99

Open
14 tasks done
Tracked by #98
nvs opened this issue Jun 29, 2022 · 1 comment
Open
14 tasks done
Tracked by #98

Revamp unpack to use position #99

nvs opened this issue Jun 29, 2022 · 1 comment
Assignees
@nvs
Labels
feature
Milestone
0.11

Comments

@nvs
Copy link
Owner

nvs commented Jun 29, 2022 •
edited
Loading

Basically, let unpack in the various file formats take an additional position argument like string.unpack does. They also need to return the updated position as well. The assertion will be removed. The intention of that particular assert was to catch updates to file formats or other issues. Unit testing should probably be implemented to catch those issues (cough #53).

  • WTS
  • INI
  • Objects
  • SLK
  • W3O
  • Units DOO
  • Doodads DOO
  • Imports
  • Cameras
  • Terrain
  • Information
  • Regions
  • Sounds
  • Pathing
@nvs nvs mentioned this issue Jun 29, 2022
Open
6 tasks
@nvs nvs changed the title Revamp unpack and pack in the various file formats. Essentially, support taking and returning position. This will require removal of the assertion comparing input size to position. Revamp unpack to use position Jun 29, 2022
@nvs nvs added this to the 0.11 milestone Jun 29, 2022
@nvs nvs mentioned this issue Jun 29, 2022
Open
@nvs nvs added the feature label Jun 29, 2022
nvs added a commit that referenced this issue Jul 4, 2022
@nvs
Objects: Add position to unpack [NOTE: Before W3O!]
a04fa0f 
Issue: #99
nvs added a commit that referenced this issue Jul 4, 2022
@nvs
SLK: Add position support to unpack
276c909 
Issue: #99
nvs added a commit that referenced this issue Jul 4, 2022
@nvs
INI: Switch to using LPEG for unpack
5663c05 
Issue: #99

Switching to LPEG should make it easier to properly support the file
format.  In particular, double quoted lists are now handled properly.
Additionally, fewer assumptions regarding values are made.

For unquoted values, everything until the EOL is now returned.  Within
the game files there are cases where trailing comments are not removed.
There are cases where they are.  How are we supposed to know?

For quoted values, both single and double, the quotes are no longer
removed.  There are cases where some values must be double quoted.
Others must be singled quoted.  Again, how are we supposed to know?

So we make less assumptions, and let the user decide.
nvs added a commit that referenced this issue Jul 4, 2022
@nvs
WTS: Add position support to unpack
1b7a741 
Issue: #99
nvs added a commit that referenced this issue Jul 4, 2022
@nvs
Objects: Add position to unpack
e2e027c 
Issue: #99
nvs added a commit that referenced this issue Jul 4, 2022
@nvs
SLK: Add position support to unpack
1f50219 
Issue: #99
nvs added a commit that referenced this issue Jul 4, 2022
@nvs
INI: Switch to using LPEG for unpack
9b63654 
Issue: #99

Switching to LPEG should make it easier to properly support the file
format.  In particular, double quoted lists are now handled properly.
Additionally, fewer assumptions regarding values are made.

For unquoted values, everything until the EOL is now returned.  Within
the game files there are cases where trailing comments are not removed.
There are cases where they are.  How are we supposed to know?

For quoted values, both single and double, the quotes are no longer
removed.  There are cases where some values must be double quoted.
Others must be singled quoted.  Again, how are we supposed to know?

So we make less assumptions, and let the user decide.
nvs added a commit that referenced this issue Jul 4, 2022
@nvs
WTS: Add position support to unpack
e84d43e 
Issue: #99
@nvs
Copy link
Owner Author

nvs commented Jul 4, 2022 •
edited
Loading

This issue probably does not need to be addressed. All of the supported file formats are processed as full files. The exception, and what prompted consideration of this change, is W3O support. Its format is a bunch of object modification files shoved together.

Instead of adjusting every file format to work in a way that will never be needed, it is probably best to do one of the following:

  • Split the input string of the W3O file a bunch of times.
  • Adjust the private object handling to take additional information (e.g. a position).

@nvs nvs closed this as not planned Won't fix, can't repro, duplicate, stale Jul 4, 2022
@nvs nvs self-assigned this Jul 4, 2022
nvs added a commit that referenced this issue Jul 5, 2022
@nvs
Objects: Add position to unpack
4c9c505 
Issue: #99
nvs added a commit that referenced this issue Jul 5, 2022
@nvs
W3O: Reintroduce support
740c38b 
Issue: #99
Issue: #100
nvs added a commit that referenced this issue Jul 5, 2022
@nvs
SLK: Add position support to unpack
2d7dc1e 
Issue: #99
nvs added a commit that referenced this issue Jul 5, 2022
@nvs
INI: Switch to using LPEG for unpack
d7c03a2 
Issue: #99

Switching to LPEG should make it easier to properly support the file
format.  In particular, double quoted lists are now handled properly.
Additionally, fewer assumptions regarding values are made.

For unquoted values, everything until the EOL is now returned.  Within
the game files there are cases where trailing comments are not removed.
There are cases where they are.  How are we supposed to know?

For quoted values, both single and double, the quotes are no longer
removed.  There are cases where some values must be double quoted.
Others must be singled quoted.  Again, how are we supposed to know?

So we make less assumptions, and let the user decide.
nvs added a commit that referenced this issue Jul 5, 2022
@nvs
WTS: Add position support to unpack
5bd3944 
Issue: #99
nvs added a commit that referenced this issue Jul 5, 2022
@nvs
IMP: TODO WORK
2a77960 
Issue: #99

Blah blah
nvs added a commit that referenced this issue Jul 5, 2022
@nvs
Units DOO: Update and improve
3d667a5 
Issue: #99

Now attempt to automatically determine if the file format is for
Reforged (1.32).  This is done by looking at the flags value.  If it is
a printable character (i.e. greater than `0x20`), then we make the
assumption that it is the latest file format.  Previously, we required
the user to supply a version table.

Furthermore, we adjust a few things, including the output format and
introducing `position` for `unpack`.  This could be considered a
breaking change.
nvs added a commit that referenced this issue Jul 5, 2022
@nvs
Doodads DOO: Update and improve
c9c6aca 
Issue: #99

As with the unit version, we now automatically detect the file format
introduced with Reforged (1.32).  This is done by looking at the byte
that either represents a doodad's flags or skin.  If this byte is a
printable character (i.e. greater than `0x20`), we assume it is a skin,
and thus the latest format.

Other changes include support of `position` in `unpack` and adjustments
to the output format.
@nvs nvs reopened this Jul 5, 2022
nvs added a commit that referenced this issue Jul 7, 2022
@nvs
SLK: Improve performance
f4ec3e0 
Issue: #99

Use of LPeg for `unpack` not only improves performance, but arguably
makes it easier to understand the logic involved.  Plus, it looks nice.
Right?  Right...?

Also support `position` for `unpack`.
nvs added a commit that referenced this issue Jul 7, 2022
@nvs
W3O: Reintroduce support
3be8451 
Issue: #99
Issue: #100
nvs added a commit that referenced this issue Jul 7, 2022
@nvs
INI: Switch to using LPEG for unpack
6debf5f 
Issue: #99

Switching to LPEG should make it easier to properly support the file
format.  In particular, double quoted lists are now handled properly.
Additionally, fewer assumptions regarding values are made.

For unquoted values, everything until the EOL is now returned.  Within
the game files there are cases where trailing comments are not removed.
There are cases where they are.  How are we supposed to know?

For quoted values, both single and double, the quotes are no longer
removed.  There are cases where some values must be double quoted.
Others must be singled quoted.  Again, how are we supposed to know?

So we make less assumptions, and let the user decide.
nvs added a commit that referenced this issue Jul 7, 2022
@nvs
Units DOO: Update and improve
29c3555 
Issue: #99

Now attempt to automatically determine if the file format is for
Reforged (1.32).  This is done by looking at the flags value.  If it is
a printable character (i.e. greater than `0x20`), then we make the
assumption that it is the latest file format.  Previously, we required
the user to supply a version table.

Furthermore, we adjust a few things, including the output format and
introducing `position` for `unpack`.  This could be considered a
breaking change.
nvs added a commit that referenced this issue Jul 7, 2022
@nvs
Doodads DOO: Update and improve
891e8dc 
Issue: #99

As with the unit version, we now automatically detect the file format
introduced with Reforged (1.32).  This is done by looking at the byte
that either represents a doodad's flags or skin.  If this byte is a
printable character (i.e. greater than `0x20`), we assume it is a skin,
and thus the latest format.

Other changes include support of `position` in `unpack` and adjustments
to the output format.
nvs added a commit that referenced this issue Jul 7, 2022
@nvs
W3S: Add new format and features
a121f2d 
Issue: #99

Format `3` of the map is now supported.  The output format for the
module has changed as well.
nvs added a commit that referenced this issue Jul 7, 2022
@nvs
W3E: Minor changes
ac51075 
Issue: #99
nvs added a commit that referenced this issue Jul 7, 2022
@nvs
W3R: Minor changes
ad5e6a2 
Issue: #99
nvs added a commit that referenced this issue Jul 7, 2022
@nvs
WPM: Minor changes
c342a69 
Issue: #99
nvs added a commit that referenced this issue Jul 7, 2022
@nvs
W3C: Update and improve
99028ec 
Issue: #99

We now automatically determine the correct file format in both `unpack`
and `pack`.  This is done by seeing if the entire input is consumed in
`unpack`, and checking for the presence of the new camera values in
`pack`.

Other changes include support of `position` in `unpack`.
nvs added a commit that referenced this issue Jul 7, 2022
@nvs
W3I: Minor changes
1ecfffc 
Issue: #99
nvs added a commit that referenced this issue Jul 7, 2022
@nvs
IMP: Minor changes
499ab0d 
Issue: #99

Note that the output format has changed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nvs nvs

Labels
feature
Projects
None yet
Milestone
0.11
Development

No branches or pull requests
1 participant
@nvs