Skip to content
Change the repository type filter

All

    Repositories list

    • disco

      Public
      DISCO is a code-free and installation-free browser platform that allows any non-technical user to collaboratively train machine learning models without sharing any private data.
      TypeScript
      Apache License 2.0
      261585512Updated Dec 23, 2024Dec 23, 2024
    • nanoGPT-like codebase for LLM training
      Python
      MIT License
      248134Updated Dec 18, 2024Dec 18, 2024
    • ML_course

      Public
      EPFL Machine Learning Course, Fall 2024
      Jupyter Notebook
      9161.3k31Updated Dec 10, 2024Dec 10, 2024
    • prefixlm

      Public
      Python
      MIT License
      0100Updated Dec 8, 2024Dec 8, 2024
    • Python
      111803Updated Nov 5, 2024Nov 5, 2024
    • Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"
      Python
      MIT License
      26500Updated Oct 30, 2024Oct 30, 2024
    • powersgd

      Public
      Practical low-rank gradient compression for distributed optimization: https://arxiv.org/abs/1905.13727
      Python
      MIT License
      3314411Updated Oct 29, 2024Oct 29, 2024
    • CoBo

      Public
      Python
      0000Updated Oct 22, 2024Oct 22, 2024
    • CoMiGS

      Public
      Python
      MIT License
      0000Updated Oct 2, 2024Oct 2, 2024
    • Exploration on-device self-supervised collaborative fine-tuning of large language models with limited local data availability, using Low-Rank Adaptation (LoRA). We introduce three distinct trust-weighted gradient aggregation schemes: weight similarity-based, prediction similarity-based and validation performance-based.
      Python
      Apache License 2.0
      0310Updated Sep 2, 2024Sep 2, 2024
    • SGD with compressed gradients and error-feedback: https://arxiv.org/abs/1901.09847
      Jupyter Notebook
      MIT License
      103022Updated Jul 25, 2024Jul 25, 2024
    • EPFL Course - Optimization for Machine Learning - CS-439
      Jupyter Notebook
      3201.2k50Updated Jun 27, 2024Jun 27, 2024
    • REQ

      Public
      Python
      Apache License 2.0
      01600Updated Jun 10, 2024Jun 10, 2024
    • CoTFormer

      Public
      Python
      MIT License
      0000Updated May 22, 2024May 22, 2024
    • Python
      0000Updated May 22, 2024May 22, 2024
    • Python
      11000Updated Apr 18, 2024Apr 18, 2024
    • Python
      Apache License 2.0
      87801Updated Apr 16, 2024Apr 16, 2024
    • DoGE

      Public
      Codebase for ICML submission "DOGE: Domain Reweighting with Generalization Estimation"
      4000Updated Feb 4, 2024Feb 4, 2024
    • Landmark Attention: Random-Access Infinite Context Length for Transformers
      Python
      Apache License 2.0
      3641881Updated Dec 20, 2023Dec 20, 2023
    • pam

      Public
      Python
      Apache License 2.0
      31400Updated Dec 9, 2023Dec 9, 2023
    • Python
      Apache License 2.0
      0400Updated Aug 18, 2023Aug 18, 2023
    • optML-pku

      Public
      summer school materials
      54400Updated Aug 4, 2023Aug 4, 2023
    • Code for Multi-Head Attention: Collaborate Instead of Concatenate
      Python
      Apache License 2.0
      2215051Updated Jun 12, 2023Jun 12, 2023
    • Jupyter Notebook
      Other
      613320Updated Jun 2, 2023Jun 2, 2023
    • difficulty-guided text summarization
      Python
      Apache License 2.0
      5500Updated May 22, 2023May 22, 2023
    • relaysgd

      Public
      Code for the paper “RelaySum for Decentralized Deep Learning on Heterogeneous Data”
      Jupyter Notebook
      MIT License
      2900Updated Apr 21, 2023Apr 21, 2023
    • Tools for experimentation and using run:ai. The aim is for these to be small self-contained utilities that are used by multiple people.
      Python
      Apache License 2.0
      0010Updated Mar 16, 2023Mar 16, 2023
    • cifar

      Public
      MLO internal cifar 10 / 100 default implementation / reference implementation. single machine, variable batch sizes, allowing maybe gradient compression. need to have clear documentation to make it easy to use, and so that we don't loose time with looking for hyperparameters. we can later keep it in sync with mlbench too, but self-contained is e…
      Python
      0001Updated Feb 8, 2023Feb 8, 2023
    • Source code for "On the Relationship between Self-Attention and Convolutional Layers"
      Python
      Apache License 2.0
      1271.1k60Updated Jan 10, 2023Jan 10, 2023
    • Python
      4715631Updated Dec 23, 2022Dec 23, 2022