Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add new feature flag to intern value string #761

Closed
wants to merge 2 commits into from
Closed

Conversation

1zg12
Copy link

@1zg12 1zg12 commented May 25, 2022

Feature request

Existing scenario

I think the INTERN_FIELD_NAMES flag is great thing to have, which is turned on by default. This would save greatly on the memory footprint especially when the message size is huge, imagine ~3.2 million position messages, all with same key like 'portfolio', 'book'.
Instead of having ~3.2 million of portfolio in parsing each batch of message in heap, with INTERN_FIELD_NAMES, it would result in only one portfolio on the string pool regardless of the message sizes, being ~3.2 million or even more.

Changes proposed

A similar feature flag could be provided, even turned on by default as well, when parsing the values.
So that, back to the ~3.2 million records example, instead of having ~3.2 million portfolio names in the heap, the similar feature flag would result in only around ~200 portfolio name (like Jason, Jackson) in the string pool.

#726

@1zg12 1zg12 closed this May 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant