Skip to content

hazelnutcloud/godot-llama-cpp

Repository files navigation

godot-llama-cpp

Run large language models in Godot. Powered by llama.cpp.



Godot v4.2 GitHub last commit GitHub License

Overview

This library aims to provide a high-level interface to run large language models in Godot, following Godot's node-based design principles.

@onready var llama_context = %LlamaContext

var messages = [
  { "sender": "system", "text": "You are a pirate chatbot who always responds in pirate speak!" },
  { "sender": "user", "text": "Who are you?" }
]
var prompt = ChatFormatter.apply("llama3", messages)
var completion_id = llama_context.request_completion(prompt)

while (true):
  var response = await llama_context.completion_generated
  print(response["text"])

  if response["done"]: break

Features

  • Platform and compute backend support:
    Platform CPU Metal Vulkan CUDA
    macOS
    Linux 🚧
    Windows 🚧 🚧
  • Asynchronous completion generation
  • Support any language model that llama.cpp supports in GGUF format
  • GGUF files are Godot resources

Roadmap

  • Chat completions support via dedicated library for jinja2 templating in zig
  • Grammar support
  • Multimodal models support
  • Embeddings
  • Vector database using LibSQL

Building & Installation

  1. Download zig v0.13.0 from https://ziglang.org/download/
  2. Clone the repository:
    git clone --recurse-submodules https://github.com/hazelnutcloud/godot-llama-cpp.git
  3. Copy the godot-llama-cpp addon folder in godot/addons to your Godot project's addons folder.
     cp -r godot-llama-cpp/godot/addons/godot-llama-cpp <your_project>/addons
  4. Build the extension and install it in your Godot project:
    cd godot-llama-cpp
    zig build --prefix <your_project>/addons/godot-llama-cpp
  5. Enable the plugin in your Godot project settings.
  6. Add the LlamaContext node to your scene.
  7. Run your Godot project.
  8. Enjoy!

License

This project is licensed under the MIT License - see the LICENSE file for details.

About

Run large language models in Godot.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published