Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: sparse horizon handler for vdf #276

Merged
merged 5 commits into from
Nov 26, 2024
Merged

Conversation

sebhoerl
Copy link
Collaborator

@sebhoerl sebhoerl commented Nov 7, 2024

The current horizon-based handler for VDF aggregates flow over N iterations for every link in every interval (for example, 5 minutes). All of this is saved as one huge block in memory and also generates large files in the output when running large-scale scenarios, for instance when using the whole network of Bavaria.

This PR introduces (experimentally) the "Sparse Horizon" handler, which works exactly as the standard version, but only saves observations that have actually taken place. So if a link in a specific time step in a specific iteration had zero flow, it will not appear in memory. This will very likely drastically decrease memory use for large-network simulations which are run with low population sampling rates.

TODO;

  • Verify that this generates the same output as the standard Horizon handler
  • Verify that reading and writing works correctly

tkchouaki and others added 3 commits November 14, 2024 10:17
* feat: Using VDFSparseHorizonHandler by default

* fix: file writing and reading

* feat: towards testing VDFSparseHorizonHandler VS VDFHorizonHandler

* chore: cleanup

* feat: comparing output events of VDFHorizonHandler and VDFSparseHorizonHandler

---------

Co-authored-by: tkchouaki <tarek.chouaki@irt-systemx.fr>
@sebhoerl sebhoerl merged commit 7ed1950 into develop Nov 26, 2024
1 check passed
@sebhoerl sebhoerl deleted the feat/vdf-sparse-horizon branch November 26, 2024 12:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants