Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove dependency on jsonpickle in pack serialization/deserialization #836

Open
mylibrar opened this issue Jun 21, 2022 · 0 comments
Open

Comments

@mylibrar
Copy link
Collaborator

mylibrar commented Jun 21, 2022

Is your feature request related to a problem? Please describe.
Now that we have integrated DataStore with DataPack, it is no longer required to rely on jsonpickle for the serialization and deserialization of DataPack/MultiPack. We should get rid of the encoding and decoding mechanism from jsonpickle and derive our own json schema of serialized pack format.

We also want to stabilize the serialization version and create some test cases to check the json schema of DataPack/MultiPack. We expect that there should be limited updates on this schema in future development once it's stabilized.

Depends on

Describe the solution you'd like

  • Update the implementation of from_string() and to_string()
    • Remove jsonpickle encoding and decoding
      • One option is to use json.loads/json.dumps instead
    • Design a stable json schema of serialized pack
    • Might need to recursively update some __getstate__ and __setstate__ methods.
  • Stabilize the serialization schema and create test cases for validation

Describe alternatives you've considered
PACK_VERSION and PACK_ID_COMPATIBLE_VERSION in version.py should be bumped up after this update.

Additional context

  • This PR should be made to the master branch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant