Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature step3 integration #45

Merged
merged 16 commits into from
Jan 16, 2024
Merged

Feature step3 integration #45

merged 16 commits into from
Jan 16, 2024

Conversation

kmilo9999
Copy link
Collaborator

  • Adding B03_plot_spectra_ov.py to the new packaging structure

@kmilo9999
Copy link
Collaborator Author

@mochell when I run the command:
python B03_plot_spectra_ov.py SH_20190502_05180312 SH_testSLsinglefile2 True
I got a lot of warnings:

self.fig.savefig(full_name, bbox_inches='tight', format='pdf', dpi=300)
save at:  ./plots//SH/SH_testSLsinglefile2/SH_20190502_05180312/B03_spectra/B03_freq_reconst_x53.pdf
/mnt/c/Projects/Github/icesat2-tracks/src/icesat2_tracks/analysis_db/B03_plot_spectra_ov.py:497: RuntimeWarning: All-NaN slice encountered
 dd_max.append(np.nanmax(dd.data))
/mnt/c/Projects/Github/icesat2-tracks/src/icesat2_tracks/analysis_db/B03_plot_spectra_ov.py:497: RuntimeWarning: All-NaN slice encountered
 dd_max.append(np.nanmax(dd.data))
/mnt/c/Projects/Github/icesat2-tracks/src/icesat2_tracks/analysis_db/B03_plot_spectra_ov.py:497: RuntimeWarning: All-NaN slice encountered
 dd_max.append(np.nanmax(dd.data))
/mnt/c/Projects/Github/icesat2-tracks/src/icesat2_tracks/local_modules/m_general_ph3.py:149: UserWarning: This figure includes Axes that are not compatible with tight_layout, so results might be incorrect.
 self.fig.savefig(full_name, bbox_inches='tight', format='pdf', dpi=300)
save at:  ./plots//SH/SH_testSLsinglefile2/SH_20190502_05180312/B03_spectra/B03_freq_reconst_x64.pdf
/mnt/c/Projects/Github/icesat2-tracks/src/icesat2_tracks/analysis_db/B03_plot_spectra_ov.py:497: RuntimeWarning: All-NaN slice encountered
 dd_max.append(np.nanmax(dd.data))
/mnt/c/Projects/Github/icesat2-tracks/src/icesat2_tracks/analysis_db/B03_plot_spectra_ov.py:497: RuntimeWarning: All-NaN slice encountered
 dd_max.append(np.nanmax(dd.data))
/mnt/c/Projects/Github/icesat2-tracks/src/icesat2_tracks/analysis_db/B03_plot_spectra_ov.py:497: RuntimeWarning: All-NaN slice encountered
 dd_max.append(np.nanmax(dd.data))
/mnt/c/Projects/Github/icesat2-tracks/src/icesat2_tracks/analysis_db/B03_plot_spectra_ov.py:497: RuntimeWarning: All-NaN slice encountered
 dd_max.append(np.nanmax(dd.data))
/mnt/c/Projects/Github/icesat2-tracks/src/icesat2_tracks/local_modules/m_general_ph3.py:149: UserWarning: This figure includes Axes that are not compatible with tight_layout, so results might be incorrect.
 self.fig.savefig(full_name, bbox_inches='tight', format='pdf', dpi=300)
save at:  ./plots//SH/SH_testSLsinglefile2/SH_20190502_05180312/B03_spectra/B03_freq_reconst_x75.pdf

Any thoughts ?

@kmilo9999 kmilo9999 marked this pull request as draft December 11, 2023 19:31
@kmilo9999 kmilo9999 changed the base branch from main to feat-step2-intg December 12, 2023 14:41
Base automatically changed from feat-step2-intg to main December 15, 2023 21:32
@kmilo9999 kmilo9999 marked this pull request as ready for review December 27, 2023 01:19
Copy link
Member

@hollandjg hollandjg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good! I've a couple of suggestions and clarifying questions, but nothing major. Thanks, Camilo!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It think we'll need a new issue to remove the upper-case letter from this file's name, because they don't conform to PEP8. We could rename this file to startup.py or similar. But do this later and in one go once the package is cleaned up.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#56

src/icesat2_tracks/analysis_db/B03_plot_spectra_ov.py Outdated Show resolved Hide resolved
Comment on lines 729 to 730
def mean_spectral_error(self, mask=None, confidence=0.95):
return wavenumber_spectrogram.mean_spectral_error(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method seems like it must be broken – wavenumber_spectrogram doesn't look to be defined anywhere. Do we use mean_spectral_error anywhere?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be a wavenumber_spectrogram class in src/icesat2_tracks/ICEsat2_SI_tools/spectral_estimates.py with a mean_spectral_error method, imported as spec in L3.

Suggested change
def mean_spectral_error(self, mask=None, confidence=0.95):
return wavenumber_spectrogram.mean_spectral_error(
def mean_spectral_error(self, mask=None, confidence=0.95):
return spec.wavenumber_spectrogram.mean_spectral_error(

That'd fix the linting problem but I'm not sure if it's used in some other file? Maybe the class doesn't need this method. @mochell

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, that'll be it - one of the imports handled by the old scripts which ran at the start of the file. Good catch, @cpaniaguam !

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cpaniaguam is right!

.github/workflows/test-B01_SL_load_single_file.yml Outdated Show resolved Hide resolved
Comment on lines 16 to 19
if return_edges:
return Gmean, k_low_limits
else:
return Gmean
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is another thing worth checking with @mochell. The type of the returned objectn of this function is not stable and this requires special handling by the caller of the function. I think it's better to make it return a tuple (or better, a nameptuple) and make the downstream logic simpler. Maybe for a separate issue?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont see rebin used in the files running the pipeline

This comment was marked as resolved.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about returning the values Gmean,none . I can modify the only line in the whole project that needs this function to retrieve two values?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the caller of this function only needs one value, wouldn't it be better to make the function return Gmean only and make the required change?

@@ -12,7 +12,7 @@
import matplotlib
#matplotlib.use('Agg')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#matplotlib.use('Agg')

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

###
#   THIS FILE IS A LOCAL FILE
#   it is not maintained via git, it contains configs specific to the machine
###
#os.environ["DISPLAY"] = "localhost:10.0"
# 14, 16,  work
#standart libraries:

Has anyone given some thought to this? Should this be moved to configs?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#67

run: python src/icesat2_tracks/analysis_db/B02_make_spectra_gFT.py SH_20190502_05180312 SH_testSLsinglefile2 True
- name: third step make_spectra
run: python src/icesat2_tracks/analysis_db/B03_plot_spectra_ov.py SH_20190502_05180312 SH_testSLsinglefile2 True
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
run: python src/icesat2_tracks/analysis_db/B03_plot_spectra_ov.py SH_20190502_05180312 SH_testSLsinglefile2 True
run: python src/icesat2_tracks/analysis_db/B03_plot_spectra_ov.py SH_20190502_05180312 SH_testSLsinglefile2 True

Maybe formatters/pre-commit can be configured to add a new line at the end of files?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let me add the formatter in another PR

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be a local setting, though.

Gmean.k[Gmean.isel(x=slice(0, 20)).mean("x").argmax().data].data * 1,
Gmean.k[Gmean.isel(x=slice(0, 20)).mean("x").argmax().data].data * 1.25,
)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rest of the file deals with plotting. I think it's sensible to move these to another file/function to abstract away complexity. For a separate issue?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#63

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. make a method out of it

@cpaniaguam cpaniaguam self-requested a review January 11, 2024 18:35
Copy link
Member

@cpaniaguam cpaniaguam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a number of cleanups and simple refactorings. I will open a separate PR to clean up one of the files (icesat2_tracks/config/IceSAT2_startup.py) with a lot of commented code.

src/icesat2_tracks/ICEsat2_SI_tools/generalized_FT.py Outdated Show resolved Hide resolved
src/icesat2_tracks/ICEsat2_SI_tools/generalized_FT.py Outdated Show resolved Hide resolved
src/icesat2_tracks/ICEsat2_SI_tools/generalized_FT.py Outdated Show resolved Hide resolved
src/icesat2_tracks/ICEsat2_SI_tools/generalized_FT.py Outdated Show resolved Hide resolved
src/icesat2_tracks/ICEsat2_SI_tools/generalized_FT.py Outdated Show resolved Hide resolved
src/icesat2_tracks/ICEsat2_SI_tools/generalized_FT.py Outdated Show resolved Hide resolved
src/icesat2_tracks/ICEsat2_SI_tools/generalized_FT.py Outdated Show resolved Hide resolved
src/icesat2_tracks/ICEsat2_SI_tools/generalized_FT.py Outdated Show resolved Hide resolved
src/icesat2_tracks/ICEsat2_SI_tools/generalized_FT.py Outdated Show resolved Hide resolved
src/icesat2_tracks/ICEsat2_SI_tools/generalized_FT.py Outdated Show resolved Hide resolved
Copy link
Member

@hollandjg hollandjg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Once Carlos' comments are handled, this looks good to me!

Comment on lines 729 to 730
def mean_spectral_error(self, mask=None, confidence=0.95):
return wavenumber_spectrogram.mean_spectral_error(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, that'll be it - one of the imports handled by the old scripts which ran at the start of the file. Good catch, @cpaniaguam !

Comment on lines 16 to 19
if return_edges:
return Gmean, k_low_limits
else:
return Gmean

This comment was marked as resolved.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Folder added to .gitignore.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Folder added to .gitignore.

@cpaniaguam cpaniaguam self-requested a review January 16, 2024 21:49
Copy link
Member

@cpaniaguam cpaniaguam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

And congrats!!

@kmilo9999 kmilo9999 dismissed hollandjg’s stale review January 16, 2024 21:51

he says it was ok to bypass him

@kmilo9999 kmilo9999 merged commit 6ac14be into main Jan 16, 2024
1 check passed
@kmilo9999 kmilo9999 deleted the feat-step3-intg branch January 16, 2024 21:51
@mochell
Copy link
Collaborator

mochell commented Jan 19, 2024

@kmilo9999 l when I run the command: python B03_plot_spectra_ov.py SH_20190502_05180312 SH_testSLsinglefile2 True I got a lot of warnings:

..

Any thoughts ?

I think its just a bunch of warmings in cases when there is no data in the slice. should be solvable with an if statement..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants