DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
-
Updated
Sep 30, 2024 - Python
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Making large AI models cheaper, faster and more accessible
LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training
SC23 Deep Learning at Scale Tutorial Material
Fast and easy distributed model training examples.
A mostly POSIX-compliant utility that scans a given interval for vampire numbers.
A state-of-the-art multithreading runtime: message-passing based, fast, scalable, ultra-low overhead
Towards Rehearsal-based Continual Learning at Scale: distributed CL with Horovod + PyTorch
飞桨大模型开发套件,提供大语言模型、跨模态大模型、生物计算大模型等领域的全流程开发工具链。
Single-node data parallelism in Julia with CUDA
The project utilizes OpenMP to implement parallelism in a large dataset by leveraging multicore processor architectures to concurrently execute code sections, optimizing performance and scalability for efficient database processing
Distributed training (multi-node) of a Transformer model
SIMD multithreaded Monte Carlo options pricer in Rust 🦀
Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
Scaling Unet in Pytorch
Scaling Unet in Tensorflow
MapReduceSimulator for Scheduling and Provisioning Algorithms
This repository provides hands-on labs on PyTorch-based Distributed Training and SageMaker Distributed Training. It is written to make it easy for beginners to get started, and guides you through step-by-step modifications to the code based on the most basic BERT use cases.
pipeDejavu: Hardware-aware Latency Predictable, Differentiable Search for Faster Config and Convergence of Distributed ML Pipeline Parallelism
Add a description, image, and links to the data-parallelism topic page so that developers can more easily learn about it.
To associate your repository with the data-parallelism topic, visit your repo's landing page and select "manage topics."