Facilitate Custom File Formats - Required For Mastodon-Git Project #329
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The "Mastodon-Git" plugin aims to efficiently store Mastodon datasets inside a git repository. Unfortunately git performs very poorly when processing a history of large binary files as produced by the default Mastodon file format. Mastodon-Git therefor implements a customized Mastodon file format that can very efficiently be processed with git.
Two classes in the Mastodon core need to be public in order for mastodon-git to implement this custom file format that can still benefit from mastodon core features:
Spot
andLink
into byte arrays (classModelSerializer
)MamutViewStateXMLSerialization
)(Mastodon-git currently has its own copies of the
ProjectLoader
andProjectSaver
classes. In the long term it would be amazing if we could add a plugin API into mastodon core. That supports the implementation of custom file formats. Or at least make theProjectLoader
and-Saver
more reusable. However in the short term it would be great if theModelSerializer
andMamutViewStateXMLSerialization
were public. This would allow us the further experiment with the mastodon-git file format, without the need to use a customized mastodon.jar file.)