Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accelerate Euclidean distance by numba #1493

Merged
merged 2 commits into from
Dec 3, 2024
Merged

Conversation

dachengx
Copy link
Collaborator

@dachengx dachengx commented Dec 3, 2024

Before you submit this PR: make sure to put all operations-related information in a wiki-note, a PR should be about code and is publicly accessible

What does the code in this PR do / what does it improve?

Nothing is changed numerically.

Can you briefly describe how it works?

Can you give a minimal working example (or illustrate with a figure)?

import os
from memory_profiler import memory_usage
import numpy as np
import numba
import matplotlib.pyplot as plt
import strax
import cutax


home = os.path.expanduser('~')
st = cutax.xenonnt_offline(xedocs_version='global_v16')
run_id = '050775'
st.storage = [strax.DataDirectory(os.path.join(home, f'{run_id}/input'), readonly=True)]

PeakletClassificationSOM0 = st.get_single_plugin(run_id, 'peaklet_classification')
peaklets = st.get_array(run_id, 'peaklets', chunk_number={'peaklets': 0}, progress_bar=False)

def compute():
    PeakletClassificationSOM0.compute(peaklets)

mem = memory_usage(proc=compute, interval=0.1, timestamps=True)
mem = np.array(mem)

fig, ax = plt.subplots(1, 1, figsize=(4, 3))
ax.plot(mem[:, 1] - mem[0, 1], mem[:, 0] / 1e3, color='black', lw=0.5)
ax.set_xlabel('Time [s]')
ax.set_ylabel('Memory usage [GB]')

fig.savefig('peaklet_classification.svg', transparent=True, dpi=800)
plt.close(fig)

The improvement is visible:
image

Please include the following if applicable:

  • Update the docstring(s)
  • Update the documentation
  • Tests to check the (new) code is working as desired.
  • Does it solve one of the open issues on github?

Notes on testing

  • Until the automated tests pass, please mark the PR as a draft.
  • On the XENONnT fork we test with database access, on private forks there is no database access for security considerations.

All italic comments can be removed from this template.

@coveralls
Copy link

coveralls commented Dec 3, 2024

Coverage Status

coverage: 89.501% (+0.04%) from 89.463%
when pulling 9438f56 on numba_euclidean_dist
into 9813b30 on master.

@LuisSanchez25
Copy link
Contributor

Looks good to me, let me just do some quick tests to make sure we get the same numbers for both methods

@dachengx dachengx marked this pull request as ready for review December 3, 2024 14:53
@LuisSanchez25
Copy link
Contributor

I have a quick question since I don't seem to be able to get the function to run locally since numba seems to reject using the assert function, did you not run into this error? The error goes away when using @numba.jit(nopython=True, cache=True) instead

@dachengx
Copy link
Collaborator Author

dachengx commented Dec 3, 2024

I have a quick question since I don't seem to be able to get the function to run locally since numba seems to reject using the assert function, did you not run into this error? The error goes away when using @numba.jit(nopython=True, cache=True) instead

My gut feeling is that the discrepancy is from different numba versions. But anyway, I moved the asssert out.

@LuisSanchez25
Copy link
Contributor

Ok then other than that I think everything looks good!

@dachengx dachengx merged commit f765c10 into master Dec 3, 2024
8 checks passed
@dachengx dachengx deleted the numba_euclidean_dist branch December 3, 2024 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants