Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Random sampling for Plot Options histogram #2763

Closed
wants to merge 2 commits into from

Conversation

pllim
Copy link
Contributor

@pllim pllim commented Mar 20, 2024

Description

This pull request is to

Since we do not use glue-native histogram directly, not sure if we can take advantage of the following upstream PRs directly:

TODO

  • There must be some subtlety I am missing here because the random sampling is not giving any performance boost on a 8GB image. 😆

Change log entry

  • Is a change log needed? If yes, is it added to CHANGES.rst? If you want to avoid merge conflicts,
    list the proposed change log here for review and add to CHANGES.rst before merge. If no, maintainer
    should add a no-changelog-entry-needed label.

Checklist for package maintainer(s)

This checklist is meant to remind the package maintainer(s) who will review this pull request of some common things to look for. This list is not exhaustive.

  • Are two approvals required? Branch protection rule does not check for the second approval. If a second approval is not necessary, please apply the trivial label.
  • Do the proposed changes actually accomplish desired goals? Also manually run the affected example notebooks, if necessary.
  • Do the proposed changes follow the STScI Style Guides?
  • Are tests added/updated as required? If so, do they follow the STScI Style Guides?
  • Are docs added/updated as required? If so, do they follow the STScI Style Guides?
  • Did the CI pass? If not, are the failures related?
  • Is a milestone set? Set this to bugfix milestone if this is a bug fix and needs to be released ASAP; otherwise, set this to the next major release milestone. Bugfix milestone also needs an accompanying backport label.
  • After merge, any internal documentations need updating (e.g., JIRA, Innerspace)? 🐱

@pllim pllim added imviz performance Performance related skip-changelog-checks changelog bot directive labels Mar 20, 2024
@pllim pllim added this to the 3.9 milestone Mar 20, 2024
@github-actions github-actions bot added the plugin Label for plugins common to multiple configurations label Mar 20, 2024
@astrofrog
Copy link
Collaborator

Using this branch with a ~4Gb dataset, the histogram seems pretty fast to me. Can you try and measure the time for the histogram to update using e.g:

%time po.stretch_hist_nbins = 300

just so we can rule out other issues such as the image viewer itself being slow.

@astrofrog
Copy link
Collaborator

On further investigation I can reproduce the issue. However it does look like you are using the glue histogram viewer beneath the hood, let me see what we can do.

@pllim
Copy link
Contributor Author

pllim commented Mar 29, 2024

Good catch on the nan stuff. I am going to close this in favor of yours. Thanks, Tom!

@pllim pllim closed this Mar 29, 2024
@pllim pllim deleted the revert-pr2735 branch March 29, 2024 18:59
pllim pushed a commit that referenced this pull request May 8, 2024
* Revert "BUG: Always downsample histogram for Imviz in Plot Options (#2735)"

This reverts commit fbef31f.

* Use glue's ability to randomly sample values when computing histograms and image statistics, and remove code to handle NaN values that caused the whole array to be loaded into memory.

* Use order='K' in ravel() to avoid copy

* Avoid calling ravel() since this causes a copy for cutouts

* Add comment to explain percentile values

* Fix issue when array len() match but .shape doesn't

* Adjust reference values for test

* Bumped minimum required version of glue-core and glue-jupyter

* Fix case where data is not a Numpy array

* Add change log
meeseeksmachine pushed a commit to meeseeksmachine/jdaviz that referenced this pull request May 8, 2024
duytnguyendtn pushed a commit to duytnguyendtn/jdaviz that referenced this pull request Jul 23, 2024
… (spacetelescope#2771)

* Revert "BUG: Always downsample histogram for Imviz in Plot Options (spacetelescope#2735)"

This reverts commit fbef31f.

* Use glue's ability to randomly sample values when computing histograms and image statistics, and remove code to handle NaN values that caused the whole array to be loaded into memory.

* Use order='K' in ravel() to avoid copy

* Avoid calling ravel() since this causes a copy for cutouts

* Add comment to explain percentile values

* Fix issue when array len() match but .shape doesn't

* Adjust reference values for test

* Bumped minimum required version of glue-core and glue-jupyter

* Fix case where data is not a Numpy array

* Add change log
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
imviz performance Performance related plugin Label for plugins common to multiple configurations skip-changelog-checks changelog bot directive
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants