Relese Note 0.1.0
CodeInferflow is a efficient inference engine based on Inferflow for code large language models (Code LLMs). With CodeInferflow, you can locally deploy popular code LLMs and efficiently use code completion in VSCode.
We build CodeInferflow with CUDA 12.4.1 on linux-x64 and windows-x64 platform. To use the pre-build binary, please makesure your CUDA version >= 12.4