TL;DR at the bottom
Note
Although I include sample configuration, this is not a replacement for reading the readme for each plugin that I mention. Please setup each of these plugins individually to ensure they're working before trying to use them all together.
How to edit Jupyter Notebooks (.ipynb files) in neovim, using molten to run code, and
load/save code cell output.
This is how I edit notebooks, and it's tailored to python notebooks. It's the best experience you can get (in my opinion), but there are some extra features you can get with other plugins that I don't use but will mention at the bottom.
> your friend sends you a jupyter notebook file
> nvim friends_file.ipynb
> you see a markdown representation of the notebook, including code outputs and images
> you edit the notebook, with LSP autocomplete, and format the code cells before running
your new code, and all the cells below it, watching each cell output update as they run
> :wq
> You send the .ipynb file, complete with your changes and the output of the code you
ran, back to your friend
neovim_notebook_demo.mp4
There are four big things required for a good notebook experience in neovim:
- Code running
- Output viewing
- LSP features (autocomplete, go to definition/references, rename, format, etc.) in a plaintext/markdown file
- File format conversion
Shocker we'll be using molten. A few configuration options will dramatically improve the notebook experience of this plugin.
-- I find auto open annoying, keep in mind setting this option will require setting
-- a keybind for `:noautocmd MoltenEnterOutput` to open the output again
vim.g.molten_auto_open_output = false
-- this guide will be using image.nvim
-- Don't forget to setup and install the plugin if you want to view image outputs
vim.g.molten_image_provider = "image.nvim"
-- optional, I like wrapping. works for virt text and the output window
vim.g.molten_wrap_output = true
-- Output as virtual text. Allows outputs to always be shown, works with images, but can
-- be buggy with longer images
vim.g.molten_virt_text_output = true
-- this will make it so the output shows up below the \`\`\` cell delimiter
vim.g.molten_virt_lines_off_by_1 = trueAdditionally, you will want to setup some keybinds (as always, change the lhs to suit your needs) to run code and interact with the plugin. At a minimum you should setup:
vim.keymap.set("n", "<localleader>e", ":MoltenEvaluateOperator<CR>", { desc = "evaluate operator", silent = true })
vim.keymap.set("n", "<localleader>os", ":noautocmd MoltenEnterOutput<CR>", { desc = "open output window", silent = true })But I'd also recommend these ones:
vim.keymap.set("n", "<localleader>rr", ":MoltenReevaluateCell<CR>", { desc = "re-eval cell", silent = true })
vim.keymap.set("v", "<localleader>r", ":<C-u>MoltenEvaluateVisual<CR>gv", { desc = "execute visual selection", silent = true })
vim.keymap.set("n", "<localleader>oh", ":MoltenHideOutput<CR>", { desc = "close output window", silent = true })
vim.keymap.set("n", "<localleader>md", ":MoltenDelete<CR>", { desc = "delete Molten cell", silent = true })
-- if you work with html outputs:
vim.keymap.set("n", "<localleader>mx", ":MoltenOpenInBrowser<CR>", { desc = "open output in browser", silent = true })One of the issues with plaintext notebooks is that you end up essentially editing a markdown file, and the pyright language server (for example) can't read a markdown file and give you information about the python code cells in it. Enter Quarto, and specifically quarto-nvim.
Quarto is a lot of things. One of those is tool for writing and publishing literate programming documents, or just any markdown document really. It's built on top of Pandoc, and so can render markdown to pdf, html, or any format that Pandoc supports.
The neovim plugin quarto-nvim provides:
- LSP Autocomplete, formatting, diagnostics, go to definition, and other LSP features for code cells in markdown documents via otter.nvim
- A code running integration with molten (written by me, so I'll provide support if there are problems/bugs) to easily run code cells (including run above, run below, run all)
- A convenient way to render the file you're working on
Sample configuration for quarto-nvim
local quarto = require("quarto")
quarto.setup({
lspFeatures = {
-- NOTE: put whatever languages you want here:
languages = { "r", "python", "rust" },
chunks = "all",
diagnostics = {
enabled = true,
triggers = { "BufWritePost" },
},
completion = {
enabled = true,
},
},
keymap = {
-- NOTE: setup your own keymaps:
hover = "H",
definition = "gd",
rename = "<leader>rn",
references = "gr",
format = "<leader>gf",
},
codeRunner = {
enabled = true,
default_method = "molten",
},
})When you configure quarto, you gain access to these functions which should be mapped to commands:
local runner = require("quarto.runner")
vim.keymap.set("n", "<localleader>rc", runner.run_cell, { desc = "run cell", silent = true })
vim.keymap.set("n", "<localleader>ra", runner.run_above, { desc = "run cell and above", silent = true })
vim.keymap.set("n", "<localleader>rA", runner.run_all, { desc = "run all cells", silent = true })
vim.keymap.set("n", "<localleader>rl", runner.run_line, { desc = "run line", silent = true })
vim.keymap.set("v", "<localleader>r", runner.run_range, { desc = "run visual range", silent = true })
vim.keymap.set("n", "<localleader>RA", function()
runner.run_all(true)
end, { desc = "run all cells of all languages", silent = true })By default, quarto only activates in quarto buffers.
We will do this with an ftplugin.
Note
In order to do this, you must make sure that quarto is loaded for markdown filetypes
(ie. if you're using lazy.nvim, use ft = {"quarto", "markdown"})
-- file: nvim/ftplugin/markdown.lua
require("quarto").activate()GCBallesteros/jupytext.nvim is a plugin
that will automatically convert from ipynb files to plaintext (markdown) files, and then
back again when you save. By default, it converts to python files, but we will configure
the plugin to produce a markdown representation.
require("jupytext").setup({
style = "markdown",
output_extension = "md",
force_ft = "markdown",
})Note
Jupytext can convert to the Quarto format, but it's slow enough to notice, on open and on save, so I prefer markdown
Because Jupytext generates markdown files, we get the full benefits of quarto-nvim when using Jupytext.
Treesitter text objects help quickly
navigate cells, copy their contents, delete them, move them around, and run code with
:MoltenEvaluateOperator.
We'll first want to define a new capture group @code_cell for the filetype we want to
run code in. Here's a very simple example for markdown, but you can do this with any
filetype you want to have a code cell in:
located in: nvim/after/queries/markdown/textobjects.scm
;; extends
(fenced_code_block (code_fence_content) @code_cell.inner) @code_cell.outerWe can now use @code_cell.inner and @code_cell.outer in the treesitter-text-objects
plugin like so, I use b, you can use whatever mappings you like:
require("nvim-treesitter.configs").setup({
-- ... other ts config
textobjects = {
move = {
enable = true,
set_jumps = false, -- you can change this if you want.
goto_next_start = {
--- ... other keymaps
["]b"] = { query = "@code_cell.inner", desc = "next code block" },
},
goto_previous_start = {
--- ... other keymaps
["[b"] = { query = "@code_cell.inner", desc = "previous code block" },
},
},
select = {
enable = true,
lookahead = true, -- you can change this if you want
keymaps = {
--- ... other keymaps
["ib"] = { query = "@code_cell.inner", desc = "in block" },
["ab"] = { query = "@code_cell.outer", desc = "around block" },
},
},
swap = { -- Swap only works with code blocks that are under the same
-- markdown header
enable = true,
swap_next = {
--- ... other keymap
["<leader>sbl"] = "@code_cell.outer",
},
swap_previous = {
--- ... other keymap
["<leader>sbh"] = "@code_cell.outer",
},
},
}
})Test it by selecting the insides of a code cell with vib, or run them with
:MoltenEvaluateOperator<CR>ib.
Saving output chunks has historically not been possible (afaik) with plaintext notebooks.
You will lose output chunks in a round trip from ipynb to qmd to ipynb. And while
that's still true, we can work around it.
Jupytext updates notebooks and doesn't destroy outputs that already exist, and Molten can both import outputs from a notebook AND export outputs from code you run to a jupyter notebook file. More details about how and when this works on the advanced functionality page.
We can make importing/exporting outputs seamless with a few autocommands:
-- automatically import output chunks from a jupyter notebook
-- tries to find a kernel that matches the kernel in the jupyter notebook
-- falls back to a kernel that matches the name of the active venv (if any)
local imb = function(e) -- init molten buffer
vim.schedule(function()
local kernels = vim.fn.MoltenAvailableKernels()
local try_kernel_name = function()
local metadata = vim.json.decode(io.open(e.file, "r"):read("a"))["metadata"]
return metadata.kernelspec.name
end
local ok, kernel_name = pcall(try_kernel_name)
if not ok or not vim.tbl_contains(kernels, kernel_name) then
kernel_name = nil
local venv = os.getenv("VIRTUAL_ENV") or os.getenv("CONDA_PREFIX")
if venv ~= nil then
kernel_name = string.match(venv, "/.+/(.+)")
end
end
if kernel_name ~= nil and vim.tbl_contains(kernels, kernel_name) then
vim.cmd(("MoltenInit %s"):format(kernel_name))
end
vim.cmd("MoltenImportOutput")
end)
end
-- automatically import output chunks from a jupyter notebook
vim.api.nvim_create_autocmd("BufAdd", {
pattern = { "*.ipynb" },
callback = imb,
})
-- we have to do this as well so that we catch files opened like nvim ./hi.ipynb
vim.api.nvim_create_autocmd("BufEnter", {
pattern = { "*.ipynb" },
callback = function(e)
if vim.api.nvim_get_vvar("vim_did_enter") ~= 1 then
imb(e)
end
end,
})Note
If no matching kernel is found, this will prompt you for a kernel to start
-- automatically export output chunks to a jupyter notebook on write
vim.api.nvim_create_autocmd("BufWritePost", {
pattern = { "*.ipynb" },
callback = function()
if require("molten.status").initialized() == "Molten" then
vim.cmd("MoltenExportOutput!")
end
end,
})Warning
This export, in conjunction with the jupytext conversion, can make saving lag the editor for ~500ms, so autosave plugins can cause a bad experience.
Note
If you have more than one kernel active this will prompt you for a kernel to choose from
The Hydra plugin allows very quick navigation and code running.
I have a detailed explanation of how to set this up on the quarto-nvim wiki. Recommend setting up treesitter-text-objects before following that.
It's very common to leave an unused expression at the bottom of a cell as a way of printing the value. Pyright will yell at you for this. Fortunately we can configure it to not do that. Just add this option to whatever existing configuration you have:
require("lspconfig")["pyright"].setup({
on_attach = on_attach,
capabilities = capabilities,
settings = {
python = {
analysis = {
diagnosticSeverityOverrides = {
reportUnusedExpression = "none",
},
},
},
},
})Molten is a multi purpose code runner, I use it in regular python files to quickly test out a line of code. In those situations, creating virtual text is obnoxious, and I'd rather have output shown in a float that disappears when I move away.
Autocommands to the rescue:
-- change the configuration when editing a python file
vim.api.nvim_create_autocmd("BufEnter", {
pattern = "*.py",
callback = function(e)
if string.match(e.file, ".otter.") then
return
end
if require("molten.status").initialized() == "Molten" then -- this is kinda a hack...
vim.fn.MoltenUpdateOption("virt_lines_off_by_1", false)
vim.fn.MoltenUpdateOption("virt_text_output", false)
else
vim.g.molten_virt_lines_off_by_1 = false
vim.g.molten_virt_text_output = false
end
end,
})
-- Undo those config changes when we go back to a markdown or quarto file
vim.api.nvim_create_autocmd("BufEnter", {
pattern = { "*.qmd", "*.md", "*.ipynb" },
callback = function(e)
if string.match(e.file, ".otter.") then
return
end
if require("molten.status").initialized() == "Molten" then
vim.fn.MoltenUpdateOption("virt_lines_off_by_1", true)
vim.fn.MoltenUpdateOption("virt_text_output", true)
else
vim.g.molten_virt_lines_off_by_1 = true
vim.g.molten_virt_text_output = true
end
end,
})Since Jupytext needs a valid notebook file to convert, creating a blank new notebook is not as easy as making an empty buffer and loading it up.
To simplify this workflow, you can define a vim user command to create an empty, but valid, notebook file and open it:
-- Provide a command to create a blank new Python notebook
-- note: the metadata is needed for Jupytext to understand how to parse the notebook.
-- if you use another language than Python, you should change it in the template.
local default_notebook = [[
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython"
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
]]
local function new_notebook(filename)
local path = filename .. ".ipynb"
local file = io.open(path, "w")
if file then
file:write(default_notebook)
file:close()
vim.cmd("edit " .. path)
else
print("Error: Could not open new notebook file for writing.")
end
end
vim.api.nvim_create_user_command('NewNotebook', function(opts)
new_notebook(opts.args)
end, {
nargs = 1,
complete = 'file'
})You can then use :NewNotebook folder/notebook_name to start a new notebook from scratch!
Compared to Jupyter-lab:
- output formats. Molten can't render everything that jupyter-lab can, specifically in-editor HTML is just not going to happen
- Markdown and latex-in-markdown rendering. Currently you can render latex, but you have to send it to the kernel. It doesn't happen automatically.
- jank. the UI is definitely worse, and sometimes images will move somewhere weird until you scroll. Molten is still relatively new, and bugs are still being ironed out.
- setup is a lot of work. I've mentioned
45 different plugins that are required to get this working and all 4 of those plugins have external dependencies.
Plugins that didn't quite make it into my workflow, but which are still really good and worth looking at.
- jupyter-kernel.nvim - this plugin adds autocomplete from the jupyter kernel, as well as hover inspections from the jupyter kernel. Me personally, I'd rather just use pyright via quarto-nvim/otter.nvim. This plugin could co-exist with the current setup, but might lead to double completions, and so you might want to disable quarto's lsp features if you choose to use this plugin
- NotebookNavigator.nvim -
a plugin for editing notebooks as a different plaintext format which defines cells using
comments in the native language of the notebook. This plugin would be used in place of
quarto-nvim, as language servers just work in a
.pyfile. I prefer to edit markdown notebooks, and the point of notebooks to me is the markdown component, and having markdown shown as comments without syntax highlighting is a deal breaker.
molten-nvim + image.nvim + quarto-nvim (+ otter.nvim) + jupytext.nvim = great notebook experience, unfortunately, it does take some time to setup.