Skip to content

Commit

Permalink
[FEA][webdatamodule]: support webdataset invocable (#501)
Browse files Browse the repository at this point in the history
## Summary
The webdataset invocables are member functions of the webdataset or
webloader class that return the the same instance their are invoked
from. Previously webdatamodule does its own computation on the epoch
length but now can rely on the user input directly using
`webdataset/webloader.with_epoch()` due to this new additional feature


## Details
_Describe your changes. You can be more detailed and descriptive here.
If it is a code change, Be sure to answer:_
  - _What is changing?_

1. Add user input webdataset invocables and apply them to the
webdataset/webloader objects upon construction.
2. Remove the computation related to epoch length and update the tests
correspondingly.
3. Update the doc to reflect the change

  - _What is the new or fixed functionality?_

(see the summary above)

  - _Why or when would someone want to use these changes?_

These invocables are parts of the webdataset usage and we have been
using them in diffdock to set up the epoch length and other properties
of the dataset and dataloader objects.

  - _How can someone use these changes?_

(see the updated README.md regarding the `invoke_wds` and `invoke_wld`
arguments on webdatamodule)


## Usage
_How does a user interact with the changed code?_
(see the updated README.md regarding the `invoke_wds` and `invoke_wld`
arguments on webdatamodule)

## Testing

Tests for these changes can be run via:
```shell
pytest -v subpackages/bionemo-webdatamodule/tests
```

**Most of the changes** to files with extensions `*.py`, `*.yaml`,
`*.yml`, `Dockerfile*` or `requirements.txt` **DO REQUIRE both `pytest-`
and `jet-` CI stages**.

- [x] Did you review the [Before your PR is "Ready for review"
section](https://github.com/NVIDIA/bionemo-framework/-/blob/dev/CONTRIBUTING.md?ref_type=heads#before-pr-ready)
before asking for review?
- [x] Did you make sure your changes have tests? Did you test your
changes locally?
- [ ] Can you add [the `SKIP_CI`
label](https://github.com/NVIDIA/bionemo-framework/-/blob/dev/CONTRIBUTING.md?ref_type=heads#skip-ci)
to your PR?
- [ ] Can you add [the `PYTEST_NOT_REQUIRED`
label](https://github.com/NVIDIA/bionemo-framework/-/blob/dev/CONTRIBUTING.md?ref_type=heads#skip-pytest)
to your PR?
- [ ] Can you add [the `JET_NOT_REQUIRED`
label](https://github.com/NVIDIA/bionemo-framework/-/blob/dev/CONTRIBUTING.md?ref_type=heads#skip-jet)
to your PR?
  • Loading branch information
DejunL authored Dec 9, 2024
1 parent 2b10d1b commit d99d24c
Show file tree
Hide file tree
Showing 4 changed files with 223 additions and 172 deletions.
Loading

0 comments on commit d99d24c

Please sign in to comment.