Skip to content

Commit

Permalink
Merge pull request #465 from holochain/2024-09-30-new-holonix-upgrade…
Browse files Browse the repository at this point in the history 
…-guide

add Holonix upgrade guide
  • Loading branch information
@pdaoust
pdaoust authored Oct 11, 2024
2 parents 0e6466d + 7698eee commit 188a5e1
Show file tree
Hide file tree
Showing 4 changed files with 42 additions and 184 deletions.
4 changes: 4 additions & 0 deletions .cspell/custom-words.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ agent-centricity
Anwaar
Automerge
automerge
binaryen
buildinputs
builtins
Brisebois
Expand Down Expand Up @@ -39,6 +40,9 @@ subl
Tauri
Ulhaq
Wahlstrom
wamr
Wasmer
wasmer
WebRTC
webview
webviews
Expand Down
195 changes: 11 additions & 184 deletions src/pages/get-started/install-advanced.md
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
---
title: Dev Tools Setup
title: Setup with Nix flakes
hide:
- toc
---

::: intro
This guide assumes that you've already followed the [quick installation guide](/get-started/) and want to learn more about the set up. It describes how to manually recreate and maintain the development environment, use your default shell and preferred code editor with Nix, explains how to install specific versions of Holochain, and discusses why we use `nix develop` in the first place.
This guide assumes that you've already followed the [quick installation guide](/get-started/) and want to learn more about the development environment.
:::

## Holonix - the Holochain app development environment
Expand All @@ -15,102 +15,28 @@ If you use the scaffolding to generate the project structure, this will already

If you want to learn more about how this setup works and how to create it manually and how to maintain it, please find all the information below.

### Holonix's usage of Nix's Flakes features
### Holonix's usage of Nix's flake feature

<<<<<<< HEAD
[Flakes](https://wiki.nixos.org/wiki/Flakes) is an experimental but well-supported feature of the Nix package manager that makes it easier to manage dependencies consistently.

As of [holochain#1863](https://github.com/holochain/holochain/pull/1863) Holonix is implemented as Holochain's [flake.nix](https://github.com/holochain/holochain/blob/develop/flake.nix) output named _#holonix_ a.k.a. _devShells.${system}.holonix_.
As of [holochain#1863](https://github.com/holochain/holochain/pull/1863) Holonix is implemented as Holochain's [flake.nix](https://github.com/holochain/holochain/blob/main/flake.nix) output named _#holonix_ a.k.a. _devShells.${system}.holonix_.
=======
[Flakes](https://wiki.nixos.org/wiki/Flakes) is an experimental but well-supported feature of the Nix package manager that makes it easier to manage dependencies consistently. [Enable flakes on your system.](https://wiki.nixos.org/wiki/Flakes#Enable_flakes_temporarily)
>>>>>>> 248f5967373e66a02b71482f9e88bb033a75e1d5
The flake-based one-liner to get you an ad-hoc Holonix shell looks like this:

```shell
nix develop github:holochain/holochain#holonix
nix develop github:holochain/holonix
```

By default, the above one-liner will give you the latest recommended release (currently from the 0.2.x series). To get an ad-hoc shell with a specific version of Holochain, use the flag `--override-input versions <version_path>`. This example gives you the next release candidate in the 0.2.x series:
The above one-liner will give you the latest version of Holochain from branch `main`. To get an ad-hoc shell with a specific version of Holochain, use the flag `--override-input versions <version_path>`.

```shell
nix develop --override-input versions "github:holochain/holochain?dir=versions/0_2_rc" "github:holochain/holochain#holonix"
nix develop --override-input holochain "github:holochain/holochain?ref=main-0.4" github:holochain/holonix
```

And this example gives you the newest weekly developer snapshot (currently from the 0.3.x series):

```shell
nix develop --override-input versions "github:holochain/holochain?dir=versions/weekly" "github:holochain/holochain#holonix"
```

Take a look at the [`versions/` folder in the `holochain/holochain` repository](https://github.com/holochain/holochain/tree/develop/versions) to find out what versions you can target. Each subfolder is a valid version to use in the above command, and the `flake.nix` file inside the subfolder shows the specific release tags that will be used for Holochain and the Lair Keystore. The launcher and scaffolding tools live in separate repos, so you will get the latest release of those tools that are available on their respective release branches referenced in the version-specific `flake.nix` file. Note that you may temporarily see older versions than the current tip of the referenced release branch because our automation updates the `flake.lock` separately, so that file is the absolute source of truth for what versions you will get.

#### Enabling Flake features on your system

At the time of writing, flakes are still considered an experimental in the nix world and thus require being enabled. This happens either ad-hoc on the command itself or permanently via Nix's configuration.

If you've completed the [quick installation guide](/get-started/), including the scaffolding example, then you'll likely already had the scaffolding configure it for you via the file at _~/.config/nix/nix.conf_.

To manually configure it via this file you can run the following commands:

```shell
mkdir -p ~/.config/nix
```
```shell
echo "extra-experimental-features = nix-command flakes" >> ~/.config/nix/nix.conf
```

To learn more, please see the [Enable flakes section on the NixOS Wiki](https://nixos.wiki/wiki/Flakes#Enable_flakes).

### The anatomy of a `flake.nix`

In the root directory of your app's code, you will either find the scaffolded one, or you can manually create the `flake.nix` file. Here's an example `flake.nix` that is inspired by the scaffolding template:

```nix
{
description = "Flake for Holochain app development";
inputs = {
versions.url = "github:holochain/holochain?dir=versions/0_1";
holochain-flake = {
url = "github:holochain/holochain";
inputs.versions.follows = "versions";
};
nixpkgs.follows = "holochain-flake/nixpkgs";
};
outputs = inputs @ { ... }:
inputs.holochain-flake.inputs.flake-parts.lib.mkFlake { inherit inputs; }
{
systems = builtins.attrNames inputs.holochain-flake.devShells;
perSystem = { config, pkgs, system, ... }: {
devShells.default = pkgs.mkShell {
inputsFrom = [
inputs.holochain-flake.devShells.${system}.holonix
];
packages = [
pkgs.nodejs-18_x
];
};
};
};
}
```

In principle a flake implements a function that produces a set of _outputs_ from a given set of _inputs_, keeping the side-effects to an absolute minimum if not at zero.

#### `inputs`

This flake declares one input named `holochain-flake` that the Holochain Github repository. This input will look for a `flake.nix` in the default branch of the remote repository.
The `versions` input of the `holochain-flake` input is explicitly specified to track the _0_1_ series, which refers to Holochain's Beta 0.1 and its compatible tools.

The flake follows (think inherits) the `nixpkgs` input of the `holochain-flake` input. This ensures that your development environment passes all the same buildinputs to the component packages, giving you very high chances to make use of our Cachix binary cache.

#### `outputs`

In the `outputs` set, this flake composes a dev shell that inherits its inputs from the `holonix` dev shell and adds the NodeJS package. To find the names of the packages you're interested in, the [nixos.org package search](https://search.nixos.org/packages?channel=unstable&) can be used.

### `flake.lock` file

Once the `flake.nix` is created (and added to the git repo), the lockfile can be initialized by running `nix flake update`. The resulting `flake.lock` records pinned references to all the `inputs` at the given point in time, in our case to the the `holochain-flake` and of all its inputs transitively; altogether keeping track of all the dependencies of your app's development environment.

### A gotcha with Flakes and Git

The behavior of `nix` commands that rely on a `flake.nix` as its input such as `nix develop` can be counterintuitive in a git repository.
Expand All @@ -119,23 +45,6 @@ Specifically, if the `flake.nix` is not tracked in git, the command will fail al

The simple solution to is to `git add flake.*` after your initial creation of your flake if you manually create a repository. In case of scaffolding a repository this is taken care of by the scaffolding process for you.

#### Updating the component versions

Each time the following command is run, it looks up the most recent revisions of the inputs and locks them in the `flake.lock` file.

```shell
nix flake update
```

If you want to only update a specific input, you can use the following command. Here it shows updating only the _holochain_ input:

```shell
nix flake lock --update-input holochain
```

_Note that if your directory is a git repository it is recommended to `git commit flake.lock` to ensure consistency between the development environment and your app's source code._


### Holonix inspection commands

Built into Holochain and holonix are a few commands that give insight about versions of Holochain components.
Expand Down Expand Up @@ -179,94 +88,12 @@ A sample output of this command looks like this (JSON formatted using `jq` i.e.
}
```

### Using a specific version of the development tools

Here's an example of how to override the inputs of the flake to pick a different version of the `holochain` component, which includes the `holochain` conductor binary and the `hc` CLI tool:

```nix
inputs = {
versions.url = "github:holochain/holochain?dir=versions/0_1";
holochain-flake = {
url = "github:holochain/holochain";
inputs.versions.follows = "versions";
inputs.holochain.url = "github:holochain/holochain/<whichever-git-branch-tag-or-commit>";
};
...
```

You can override the versions of four different Holochain components: `holochain`, `lair`, `launcher`, and `scaffolding`. The `inputs.versions.url` field points to a file in the `holochain/holochain` GitHub repo containing versions of each of these, which are known to be mutually compatible. As you can see in the snippet above, the URLs of any of those components can be overridden. Take a look at the [versions file](https://github.com/holochain/holochain/blob/develop/versions/0_1/flake.nix) for an example of how we specify their URLs using Git tags.

_Note that by specifying custom component URLs, you will probably get a binary cache miss when entering the shell, and it will have to compile the custom component versions ad-hoc._

### Using your default `$SHELL`

Many developers have their shell set up just the way they like it, whether a custom-formatted prompt or a completely different shell such as `zsh` and `fish`. If you don't want Holonix to clobber your carefully-crafted environment, try adding `--command $SHELL` to the end of your `nix develop` command:

```shell
nix develop github:holochain/holochain#holonix --command "$SHELL"
```

### Using your favorite text editor or IDE

In most cases you can run your editor as you normally would. However, if you are using a text editor or integrated development environment (IDE) that needs to communicate with the Rust compiler for real-time syntax checks, then you should launch it from inside the `nix develop`. This is because Holonix comes with its own version of Rust that might be different from what you may already have installed.

To do this, just open your editor from the command line while you are in the `nix develop` (this example uses Vim):

```shell
nix develop github:holochain/holochain#holonix
```
```shell
cd my_project
```
```shell
vim my_file.rs
```

## More info on Nix

We use the Nix/NixOS toolkit to build consistent development, testing, and deployment environments for Holochain Core and apps. It consists of two systems:

* NixOS, a tool for reliably building Linux-based systems from a set of configuration files (we use NixOS in our HoloPorts and automated testing VMs)
* Nix, a package manager that works on many OSes and uses the same configuration file format as NixOS

The main components of the tooling for Holochain development are:

* The [Rust](https://rust-lang.org) programming language
* [Node.JS](https://nodejs.org) and [npm](https://npmjs.com)
* Cryptographic libraries
* Common automations and scripts

It is important that these remain consistent, so you can get your work done without fighting package and compiler issues. And when it comes time to compile and distribute your application, it's **very important to have a deterministic build system** so the same DNA source code always results in the same hash.

The main Nix tool used in Holochain development workflows is `nix develop`, a program that overlays a new Bash environment and set of tools on top of your existing shell environment.

The full suite of Nix tooling is broad and deep. There's even a dedicated programming language, called [Nix expressions](https://nixos.org/manual/nix/stable/#functional-package-language). Learn more with the [NixOS Wiki](https://nixos.wiki) or the [Pills](https://nixos.org/nixos/nix-pills/) Tutorial. The [NixOS community chat on matrix.to](https://matrix.to/#/#community:nixos.org) is active and helpful.

### `nix develop`

While working on Holochain, you will usually have an active `nix develop` to run commands. This shell overlays Holochain-specific configuration on top of your existing shell - environment variables, Rust toolchains, binaries, libraries, and development tools - giving you a consistent development environment to build Holochain apps. The shell environment is only set up in the current shell session, and will be reset automatically when you `exit` the shell.

If you want to re-enter the shell to do more work, or create multiple terminals to work in, you'll need to re-enter the `nix develop` in each new instance. The packages are cached locally on your machine, so they will be ready the next time you enter the shell. You do need to get the package configuration files from somewhere, though. If you use the Holochain repo cloning method, they're cached on your machine too, but the ['quick install'](/get-started/) and ['using a specific version'](#using-a-specific-version-of-the-development-tools) methods require an internet connection every time you want to enter the shell.

## Uninstalling Nix

You usually don't need to uninstall anything, because `nix develop` leaves your familiar user environment alone and makes all of its own changes disappear once you exit the shell. But it does keep binaries and other packages on your device. On macOS it adds users and a user group too. If you want to free up some space, run these commands:

```shell
nix-collect-garbage -d
```

If you want to uninstall Nix as well, run these commands (you might need root privileges for the first line):

```shell
rm -rf /nix
```
```shell
rm ~/.nix-profile
```

[Detailed uninstallation instructions for macOS](https://gist.github.com/chriselsner/3ebe962a4c4bd1f14d39897fc5619732#uninstalling-nix)

## Fixing the SUID sandbox error in Ubuntu 24.04

Ubuntu 24.04 [introduced an AppArmor security policy](https://discourse.ubuntu.com/t/ubuntu-24-04-lts-noble-numbat-release-notes/39890#unprivileged-user-namespace-restrictions-15) that causes `hc spin`, which is used to test applications and their UIs, to fail with a fatal error. If you have a `package.json` that lists `@holochain/hc-spin` as a dev dependency, you may see this error message:
Expand Down
1 change: 1 addition & 0 deletions src/pages/resources/upgrade/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@ Upgrading between versions of Holochain can be a bit tricky! While Holochain is

* [Holochain Upgrade 0.1 → 0.2](/resources/upgrade/upgrade-holochain-0.2/)
* [Holochain Upgrade 0.2 → 0.3](/resources/upgrade/upgrade-holochain-0.3/)
* [Upgrading to the new Holonix](/resources/upgrade/upgrade-new-holonix/) (all Holochain versions)
26 changes: 26 additions & 0 deletions src/pages/resources/upgrade/upgrade-new-holonix.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
title: Upgrade to the new Holonix
---

::: intro
In the third quarter of 2024, we released a new [Holonix](/get-started/install-advanced/#holonix-the-holochain-app-development-environment), our development environment based on [Nix](https://nixos.org) that gives you all the dependencies to build hApps.

The new Holonix is simpler and more modular, which means that it's easier to configure it for your needs and preferences.
:::

## Upgrade an existing project

The previous and the new Holonix distributions are both based on Nix's [flakes](https://wiki.nixos.org/wiki/Flakes) feature. Fortunately, this means it's easy to upgrade. First back up and remove your project's existing flake files:

```bash
mv flake.nix flake.nix.backup
mv flake.lock flake.lock.backup
```

Then create a new flake file:

```bash
nix flake init -t github:holochain/holonix?ref=main-0.3
```

For most projects, that's all you need! If you have specific needs, such as targeting a specific Holochain version or release series, specifying build flags, or adding more Nix packages, read the [Holonix readme](https://github.com/holochain/holonix/tree/main/README.md).

0 comments on commit 188a5e1

Please sign in to comment.