server : add speculative decoding support (#10455) · 3Simplex/llama.cpp@9ca2e67 · GitHub

Commit

server : add speculative decoding support (ggerganov#10455)

Browse files

* server : add speculative decoding support

ggml-ci

* server : add helper function slot.can_speculate()

ggml-ci

Loading branch information

ggerganov authored Nov 25, 2024

1 parent 5931c1f commit 9ca2e67