DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
-
Updated
Sep 30, 2024 - Python
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Optimizing inference proxy for LLMs
Efficient global optimization toolbox in Rust: bayesian optimization, mixture of gaussian processes, sampling methods
[NeurIPS'24] Multilinear Mixture of Experts: Scalable Expert Specialization through Factorization
Mixture of Experts for Single Cell Perturbation Prediction
Fast Inference of MoE Models with CPU-GPU Orchestration
[NeurIPS 2024] RealCompo: Balancing Realism and Compositionality Improves Text-to-Image Diffusion Models
Surrogate Modeling Toolbox
GraphMETRO: Mitigating Complex Graph Distribution Shifts via Mixture of Aligned Experts (NeurIPS 2024)
Awesome Mixture of Experts (MoE): A Curated List of Mixture of Experts (MoE) and Mixture of Multimodal Experts (MoME)
Training two separate expert neural networks and one gater that can switch the expert networks.
Tutel MoE: An Optimized Mixture-of-Experts Implementation
Continuous Generalist Web Navigation Agent using domain-wise Mixture-of-Experts that merges individually fine-tuned LLMs as domain experts.
Implementation of the "the first large-scale multimodal mixture of experts models." from the paper: "Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts"
Implementation of Switch Transformers from the paper: "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity"
The official implementation of the paper "Demystifying the Compression of Mixture-of-Experts Through a Unified Framework".
Self contained pytorch implementation of a sinkhorn based router, for mixture of experts or otherwise
A library for easily merging multiple LLM experts, and efficiently train the merged LLM.
Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding (Findings of EMNLP'23)
Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind
Add a description, image, and links to the mixture-of-experts topic page so that developers can more easily learn about it.
To associate your repository with the mixture-of-experts topic, visit your repo's landing page and select "manage topics."