Fix remaining swarm D->H->D copies #1145
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
brryan
commented
Jul 31, 2024
brryan
commented
Jul 31, 2024
brryan
changed the title
lroberts36
approved these changes
Aug 1, 2024
pgrete
approved these changes
Aug 8, 2024
…swarm_prefix_sums
…enon into brryan/more_swarm_prefix_sums
* First kernel down * Further cleanup * Notes * Send seems to work * Need to promote particles example to latest pattern * May have fixed particles example * cycles... * iterative tasking is only iterating on one meshblock in particles example... * Still not working... * New loop seems to work * Cleaned up some printfs/marked unused code for deletion * New algorithm is cycling, time to clean up * Fixed indexing bug... * Clean up * finished_transport shouldnt be provided by swarm * Reverting to old manual iterative tasking * Still working... * Starting to make progress... * Cleaned up * format * A few leftover print statements
|
OK I fixed up the swarm communication logic to also function on device. Those specific changes are shown in #1154. I used a combination of parallel |
brryan
commented
Aug 20, 2024
|
OK this is ready for review/merge and should cure the current host/device copy performance bottlenecks in swarms. |
Yurlungur
approved these changes
Aug 20, 2024
|
@AstroBarker now that you're back you should be paying attention to swarm development in parthenon. |
pgrete
reviewed
Aug 26, 2024
…enon into brryan/more_swarm_prefix_sums
|
I did some quick downstream tests and the results look good.
Feel free to hit merge whenever. |
…enon into brryan/more_swarm_prefix_sums
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Summary
There are some outstanding places in the particle infrastructure where we copy arrays to host to perform sorting logic. This is almost certainly responsible for a just uncovered major performance bottleneck downstream. Should be fixable with prefix sums on device. This PR aims to do that.
Swarmconstructor with kernel launch rather than device/host copiesSwarmmethod names from camel case to uppercaseDefragmentmethod to use kernel launches rather than device/host copies to defragment the memory pool of particlesAddEmptyParticlesmethod to use kernel launches rather than device host copies to provide a list of empty particle indicesnull_ptr-ness of user swarm boundary functions that was not interacting correctly with recent changes to the forest infrastructure.swarm_comms.cpp(can possibly defer this to another PR if anyone wants to merge this before middle of next week)AddRegionto prevent segfault if reference is not properly stored when making a task collection.PR Checklist