v2.3.0

🦚  New Features

• em27.PROFFAST_MULTIPLIERS, em27. PROFFAST_UNITS, ...
  Add EM27/SUN constants for instrument colors, serial numbers, data units, etc.
  see in docs

• plotting.add_colorpatch_legend
  A function to add a "Colorpatch Legend" to a plot
  see in docs

• RandomLabelGenerator
  Generate random names - like Docker container names curious-einstein, wild-picard and so on
  see in docs

• tum_esm_utils.timing.parse_iso_8601_datetime
  Added because datetime.datetime.fromisoformat does not support all ISO 8601 timezone variants
  see in docs

• tum_esm_utils.files.list_directory
  Adds more options the os.listdir method - only include directories/files, filter by regex, ignore by Unix shell patterns
  see in docs

• tum_esm_utils.files.render_directory_tree
  Render a directory tree as a string
  see in docs

• tum_esm_utils.text.simplify_string_characters
  Simplify a string by replacing special characters with their ASCII counterparts and removing unwanted characters with. For example, simplify_string_characters("Héllo, wörld!") will return "hello-woerld".
  see in docs

• tum_esm_utils.text.replace_consecutive_characters
  Remove consecutive whitespaces or other characters with tum_esm_utils.text.replace_consecutive_characters
  see in docs

🦀  Dependency Updates

• Upgrade minimum Python version to 3.10. This follows SPEC 0 — Minimum Supported Dependencies recommendations.

v2.2.0

Upgrade Numpy (v1 to v2) and Polars (v0 to v1).

Add Version validator (#19).

v2.1.0

This release brings a significantly improved opus file validation based on Proffast 2.4. The old version could only handle a few reasons of file corruption. Furthermore the output did not list the files that the preprocessing could not even read.

Old validator:

detection_results = tum_esm_utils.em27.detect_corrupt_ifgs(
    "/path/to/a/folder/with/interferograms"
)
assert detection_results == {
    "md20220409s0e00a.0199": [
        "charfilter 'GFW' is missing",
        "charfilter 'GBW' is missing",
        "charfilter 'HFL' is missing",
        "charfilter 'LWN' is missing",
        "charfilter 'TSC' is missing",
    ]
}

New validator:

detection_results = tum_esm_utils.em27.detect_corrupt_opus_files(
    "/path/to/a/folder/with/interferograms"
)

detection_results == {
    'md20220409s0e00a.0199': [
        'Charfilter "DUR" is missing',
        'Charfilter "GBW" is missing',
        'Charfilter "GFW" is missing',
        'Charfilter "HFL" is missing',
        'Charfilter "LWN" is missing',
        'Charfilter "TSC" is missing',
        'Differing sizes of dual channel IFGs!',
        'IFG size too small!',
        'Inconsistent dualifg!',
        'Inconsistent parameter kind in OPUS file!'
    ],
    'comb_invparms_ma_SN061_210329-210329.csv': [
        'File not even readible by the parser'
    ],
    'comb_invparms_mc_SN115_220602-220602.csv': [
        'File not even readible by the parser'
    ],
    'md20220409s0e00a.0200': [
        'File not even readible by the parser'
    ]
}

The function tum_esm_utils.em27.detect_corrupt_ifgs has been renamed to tum_esm_utils.em27.detect_corrupt_opus_files but is of course still available under the old name too (until the next breaking release).

v2.0.0

This breaking release is necessary to make the polars library an optional dependency. Until now, every codebase that wanted to use the tum-esm-utils library would have to use the same major release of polars.

The library now has two optional modules: em27 and plotting, for which the dependencies (polars and matplotlib are not automatically installed). Install them using:

pip install "tum-esm-utils[polars]"
pip install  "tum-esm-utils[plotting]"
pip install  "tum-esm-utils[polars,plotting]"

Due to this change, the base library is kept at ~12MB, including all (sub-)dependencies, and only if you want the plotting and EM27 extras, this grows to ~174MB.

🦞  Breaking changes

I included many breaking changes here, so there will not be a need for another breaking release soon.

• Module interferograms has been renamed em27 and made optional
• Removed the context and testing modules: functionality has been moved into timing and files
• Renamed module github to code
• Removed unused functionality and changed some return types (see #18 for complete list)
• Dropped Python3.8 support because Matplotlib also dropped it
• Changed from MIT to GPL license

🦚  New features

• Added new optional plotting module
• Refactored function to load Proffast2 output files
• Added function to code module to load files from GitLab

v1.9.0

🦚  New Features

• Add a new method tum_esm_utils.files.read_last_n_lines (docs)
• Add a new method tum_esm_utils.timing.parse_timezone_string (docs)

v1.8.0

🦚  New Features

• Add a new method tum_esm_utils.timing.date_range: https://tum-esm-utils.netlify.app/api-reference#date_range-1

🦀  Dependency Updates

• Update polars@0.19 to 0.20

🐜  Internal Changes

• Copy all the functions in the context module (all time related) to the new timing module
• Add deprecation warnings with the link to the new timing module to all migrated functions in the context module
• Use timing.date_range inside text.date_range (the latter is based on date strings like YYYYMMDD)

1.7.0

🦚  New Features

• Add a new method to strictly validate paths using Pydantic root models: https://tum-esm-utils.netlify.app/example-usage#strict-path-validation-with-pydantic
• Add a new very clean method to convert paths relative to the script into absolute files: https://tum-esm-utils.netlify.app/example-usage#get-absolute-paths
• Add a function to set alarms in order to catch infinite loops in your code: https://tum-esm-utils.netlify.app/example-usage#set-alarms-to-catch-infinite-loops

All of the above is tested, statically typed and documented of course 🐝

1.6.0 - Significantly Improved Documentation

We moved from Docsify to Nextra to render the actual docs page. We moved from lazydocs to pydoc-markdown to automatically generate the markdown API reference from the codebase

Feel free to use this documentation setup in your own projects to improve the usability of the content you produce 🌈

API reference - Old | New - now all on one scrollable page

Examples - Old | New - more and improved usage examples

Search - Old | New - a search that is actually useful