Skip to content

A PyTorch data loader in C++ with Python binding.

License

Notifications You must be signed in to change notification settings

zjlww/torchdata-xx

Repository files navigation

TODO List

  • (Done) Run some tutorial examples.
  • (Done) Save PyTorch item, and print out in C++.
  • (Done) Save PyTorch shard, and print out in C++.
  • (Done) File size benchmark vs pickle.
  • (Done) Migrate and optimize old code. Implement following functions:
    • (Done) Save shard in Python.
    • (Done) Load shard in C++ as dataset.
    • (Done) Access sharded dataset in Python by pybind11.
    • (Done) Remove the current package stuff!
    • (Done) Make all the functionals class types.
  • (Done) Send C++ item to PyTorch.
  • (Done) Send PyTorch item to C++.
  • (Done) Implement one function as a class, pass it to Python side, and pass back to C++ side to map a data sampler.
  • Implement all LibriTTS dataloaders in Python.
    • Build all non-sharding datasets.
    • Build sharding datasets.

About

A PyTorch data loader in C++ with Python binding.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published