Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updating pruning func for varlingam for faster processing #178

Merged
merged 1 commit into from
Apr 5, 2024

Conversation

svenpieper
Copy link
Contributor

@svenpieper svenpieper commented Apr 4, 2024

The recent update replaces a previous loop structure with a more efficient use of advanced indexing to fill the variables obj and exp. This improves code readability and may speed up execution, especially with large datasets.

Instead of iterating through each element block in blocks, a 3D array structure is now created to hold flattened ancestor indices for each block. This is done by extracting the ancestor indices for each block using blocks[:, 0, ancestor_indexes] and shaping them into a flat structure ancestor_indexes_flat.

Populating the obj variable is also optimized using advanced indexing, directly copying blocks[:, 0, i] into obj for more efficient assignment.

For the exp variable, advanced indexing is used similarly. Flattened ancestor indices are inserted into exp[:, :causal_order_no], and remaining elements from blocks[:, 1:].reshape(len(blocks), -1) are placed into the corresponding position in exp[:, causal_order_no:]. This update improves code clarity and enhance execution speed drastically.

During my research I noticed that the pruning takes a very long time and this happens especially with longer data sets with many nodes, this should fix this issue. Atleast for Varlingam.

Signed-off-by: Sven <sven@pop-os.localdomain>
Copy link
Collaborator

@kunwuz kunwuz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much! It looks great! :)

@kunwuz kunwuz merged commit 36b0829 into py-why:main Apr 5, 2024
2 checks passed
@svenpieper svenpieper deleted the fast_pruning_varlingam branch April 8, 2024 08:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants