ollama completion does not work at all

[OneOfOne]

Using manual completion (C-x) it just spams "Completion started" until I have to close nvim.

my config:

{
	'maxwell-bland/cmp-ai',
	config = function()
		local cmp_ai = require('cmp_ai.config')

		cmp_ai:setup({
			max_lines = 100,
			provider = 'Ollama',
			provider_options = {
				stream = true,
				model = 'mistral',
			},
			notify = true,
			notify_callback = function(msg)
				vim.notify(msg)
			end,
			run_on_every_keystroke = false,
		})
	end,
},

[tzachar]

Did you install Ollama?

[OneOfOne]

Yes, it works fine with the chatgpt plugin for chatting, and I can use it from the terminal, I tried several models so far none of them worked

[tzachar]

try setting model = codellama
other than that, it works on my end...

[RyanGreenup]

I was having some issues at first as well, however, I think I wasn't waiting long enough and eventually it does show up (although sometimes it doesn't when I'm low on memory).

@OneOfOne if it helps, here is the configuration that worked for, hitting C-x generates the completion (without streaming so give it 5-60s)

• added cmp_ai (3a5f7355) · Commits · RyanGreenup / dotfiles · GitLab

[bruteforks]

same issue. Completion started, but no VRAM usage via gpustat -cp --watch.
Ollama works fine locally, and works using Gen plugin.

Must be a config issue. I'll try to reproduce it with a minimal config.

---

Edit: it was user error. I feel very stupid, but I'm going to leave stuff here to maybe help someone in the future.

I mistakenly thought codellama:latest was unnecessary because it has the same id as codellama:instruct, and so I removed it.

I was trying to use codellama:code as the provider, and that was the issue. I redownloaded codellama which pulled the cached codellama:latest, and switched to codellama as the provider as tzachar suggested above.

Before going overboard and creating a minimal config like I did, make sure to do a very quick test using curl to debug.

curl http://localhost:11434/api/generate -d '{
"model": "codellama",
"prompt": "Why is the sky blue?"
}'

---

and the not-so-minimal config with lazy, nvim -u lazy_min.lua main.py:

-- Set leader key
vim.g.mapleader = " "

-- Install lazy.nvim if not already installed
local lazypath = vim.fn.stdpath("data") .. "/lazy/lazy.nvim"
if not vim.loop.fs_stat(lazypath) then
	vim.fn.system({
		"git",
		"clone",
		"--filter=blob:none",
		"https://github.com/folke/lazy.nvim.git",
		"--branch=stable", -- latest stable release
		lazypath,
	})
end
vim.opt.rtp:prepend(lazypath)

vim.lsp.set_log_level("debug")
vim.g.cmp_ai_debug = true

-- Install plugins
require("lazy").setup({
	-- nvim-cmp and dependencies
	{ "neovim/nvim-lspconfig" },
	{ "hrsh7th/nvim-cmp" },
	{ "hrsh7th/cmp-nvim-lsp" },
	{ "tzachar/cmp-ai", dependencies = { "nvim-lua/plenary.nvim" } },
	{ "L3MON4D3/LuaSnip" },
	{ "saadparwaiz1/cmp_luasnip" },
	{ "onsails/lspkind.nvim" },
	{
		"folke/noice.nvim",
		event = "VeryLazy",
		opts = {
			lsp = {
				override = {
					["vim.lsp.util.convert_input_to_markdown_lines"] = true,
					["vim.lsp.util.stylize_markdown"] = true,
					["cmp.entry.get_documentation"] = true, -- requires hrsh7th/nvim-cmp
				},
			},
			presets = {
				bottom_search = true, -- use a classic bottom cmdline for search
				command_palette = true, -- position the cmdline and popupmenu together
				long_message_to_split = true, -- long messages will be sent to a split
				inc_rename = false, -- enables an input dialog for inc-rename.nvim
				lsp_doc_border = false, -- add a border to hover docs and signature help
			},
		},
		dependencies = {
			"MunifTanjim/nui.nvim",
			"rcarriga/nvim-notify",
		},
	},
})

-- Basic Neovim options
vim.opt.completeopt = { "menu", "menuone", "noselect" }

-- Setup nvim-cmp
local cmp = require("cmp")
cmp.setup({
	debug = true,
	snippet = {
		expand = function(args)
			require("luasnip").lsp_expand(args.body)
		end,
	},
	mapping = cmp.mapping.preset.insert({
		["<C-b>"] = cmp.mapping.scroll_docs(-4),
		["<C-f>"] = cmp.mapping.scroll_docs(4),
		["<C-Space>"] = cmp.mapping.complete(),
		["<C-e>"] = cmp.mapping.abort(),
		["<CR>"] = cmp.mapping.confirm({ select = true }),
		["<C-x>"] = cmp.mapping(
			cmp.mapping.complete({
				config = {
					sources = cmp.config.sources({
						{ name = "cmp_ai" },
					}),
				},
			}),
			{ "i" }
		),
	}),
	sources = cmp.config.sources({
		{ name = "cmp_ai" },
		{ name = "nvim_lsp" },
		{ name = "luasnip" },
	}),
	formatting = {
		format = function(entry, vim_item)
			local source_names = {
				nvim_lsp = "[LSP]",
				cmp_ai = "[AI]",
				luasnip = "[Snippet]",
			}
			vim_item.menu = source_names[entry.source.name]
			return vim_item
		end,
	},
})

-- Setup lspconfig
local capabilities = require("cmp_nvim_lsp").default_capabilities()
local lspconfig = require("lspconfig")
lspconfig.pyright.setup({
	capabilities = capabilities,
})

local cmp_ai = require("cmp_ai.config")

cmp_ai:setup({
	max_lines = 100,
	provider = "Ollama",
	provider_options = {
		model = "codellama",
	},
	notify = true,
	notify_callback = function(msg)
		vim.notify(msg)
	end,
	run_on_every_keystroke = true,
	ignored_file_types = {
	},
})

local lspkind = require("lspkind")

local source_mapping = {
	buffer = "[Buffer]",
	nvim_lsp = "[LSP]",
	nvim_lua = "[Lua]",
	cmp_ai = "[AI]",
	path = "[Path]",
}

require("cmp").setup({
	sources = {
		{ name = "nvim_lsp" },
		{ name = "luasnip" },
		{ name = "buffer" },
		{ name = "cmp_ai" },
	},
	formatting = {
		format = function(entry, vim_item)
			-- if you have lspkind installed, you can use it like
			-- in the following line:
			vim_item.kind = lspkind.symbolic(vim_item.kind, { mode = "symbol" })
			vim_item.menu = source_mapping[entry.source.name]
			if entry.source.name == "cmp_ai" then
				local detail = (entry.completion_item.labelDetails or {}).detail
				vim_item.kind = ""
				if detail and detail:find(".*%%.*") then
					vim_item.kind = vim_item.kind .. " " .. detail
				end

				if (entry.completion_item.data or {}).multiline then
					vim_item.kind = vim_item.kind .. " " .. "[ML]"
				end
			end
			local maxwidth = 80
			vim_item.abbr = string.sub(vim_item.abbr, 1, maxwidth)
			return vim_item
		end,
	},
})

local compare = require("cmp.config.compare")
cmp.setup({
	sorting = {
		priority_weight = 2,
		comparators = {
			require("cmp_ai.compare"),
			compare.offset,
			compare.exact,
			compare.score,
			compare.recently_used,
			compare.kind,
			compare.sort_text,
			compare.length,
			compare.order,
		},
	},
})

[undying]

I also tried to set it up, and it doesn't work for me. With Wireshark, I looked at request-responses that goes to the codellama model and there is a problem either in the prompt or in the prompt + response.
As a prompt, the entire file flies away, and as an answer, not the code completion, but its explanation.

Here are examples

request

{
    "options": {"temperature": 0.2},
    "model": "codellama",
    "stream": false,
    "prompt": "<PRE> from django import forms\nfrom django.core.validators import RegexValidator\n\n\nphone_number_validator = RegexValidator...(removed for readability)...\n\nclass ContactForm(forms.Form):\n    name = forms.CharField(max_length=255)\n    email = forms.EmailField()\n    message = forms.CharField(widget=forms.Textarea(), required=True)\n\nclass ContactForm(f <SUF>) <MID>"
}

response

{
    "model":"codellama",
    "created_at":"2024-07-14T13:07:24.099798Z",
    "response":"  ```\nfrom django import forms\nfrom django.core.validators import RegexValidator...(removed for readability)...\n\nclass ContactForm(forms.Form):\n    name = forms.CharField(max_length=255)\n    email = forms.EmailField()\n    message = forms.CharField(widget=forms.Textarea(), required=True)\n  ```\n  \n  This is a Django form that includes three fields: `title`, `about`, and `url`. The `phone` field has a validator to check if the input is a correct phone number, and the `barcode` field has a validator to check if the input is a correct barcode. The `ReviewForm` class includes only one field, which is a textarea for the review body. The `ContactForm` class includes three fields: `name`, `email`, and `message`.\n\nIt's important to note that in order to use this form in a Django view, you need to import it and create an instance of it. For example:\n```\nfrom .forms import NodeCreateForm\n\ndef my_view(request):\n    form = NodeCreateForm()\n    return render(request, 'my_template.html', {'form': form})\n```\nIn this example, the `NodeCreateForm` class is imported and an instance of it is created and passed to the template as a context variable called `form`. The template can then use this variable to display the form and handle the user input.",
    "done":true,"done_reason":"stop",
    "context":[...],
    "total_duration":15330468416,
    "load_duration":9605625,
    "prompt_eval_count":420,
    "prompt_eval_duration":108984000,
    "eval_count":633,
    "eval_duration":15163806000
}

I don't know how it supposed to work in the right way, but I guess that it should reply with code completion snippet rather than with description.

[gmatht]

Not sure what the issue is. Tabby seems to work fine.

[danielb2]

anyone having luck with this?