GPU programming is still too hard, and the current LLMs aren't helping much. We think we can build something better, an LLM that can actually write good GPU code.
We're doing this in public - all our training runs, conversations, and infrastructure will be open.
We mostly talk on discord.gg/gpumode in the popcorn channel.
We're collecting and synthetically generating as many GPU kernels as we can.
We need compute - we're building tools to track progress and share results in real-time through Discord.
We're investigating how to prompt, train, deploy, and sample LLMs for effective GPU code generation.
We're exploring new abstractions to simplify GPU programming.
- KernelBench: https://github.com/ScalingIntelligence/KernelBench
- Discord Cluster Manager: https://github.com/gpu-mode/discord-cluster-manager
- ThunderKittens: https://github.com/HazyResearch/ThunderKittens
Our collaborators
- https://scalingintelligence.stanford.edu/
- https://hazyresearch.stanford.edu/
- https://pytorch.org/
- https://discord.com/invite/gpumode
Our compute sponsors
And anyone who shares our goals is welcome to join