Skip to content

Latest commit

 

History

History
405 lines (297 loc) · 16.4 KB

README.md

File metadata and controls

405 lines (297 loc) · 16.4 KB
Raw
copyright permalink
Copyright (c) Runtime Verification, Inc. All Rights Reserved.
README.html

Join the chat on Matrix

Introduction

The K Framework is a tool for designing and modeling programming languages and software/hardware systems. At the core of the K Framework is a programming, modeling, and specification language called K. The K Framework includes tools for compiling K specifications to build interpreters, model checkers, verifiers, associated documentation, and more.

Quick Start

If you are not a K developer, but just want to get started using K, we provide a streamlined installation process for any system that supports Nix:

bash <(curl https://kframework.org/install)
kup install k

For more information on the kup tool and other packaged releases of K, please refer to our installation notes.

Preface

This is a readme file for K developers. Users should feel comfortable using the command line, as we do not provide GUI tools at this time.

K-based tool users should:

  1. Consult their tool documentation for build/installation instructions.
  2. If needed, download a packaged release of the K Framework as part of their tool setup process.

If you are interested in quickly trying out the K Framework without building from source, please see our packaged release installation guide.

The rest of this file assumes you intend to build and install the K Framework from source.

Note that the K Framework can only be built on (x86-64) Linux-like systems, e.g., this also includes macOS/brew (x86-64) as well as the Windows Subsystem for Linux. All 32-bit systems are not supported. See the installation notes for details about supported configurations and system setup.

Contents

  1. Prerequisite Install Guide
  2. Build and Install Guide
  3. IDE Setup
  4. Running the Test Suite
  5. Changing the KORE Data Structures
  6. Building the Final Release Directory/Archives
  7. Compiling Definitions and Running Programs
  8. Installing Python Support
  9. Troubleshooting

Prerequisite Install Guide

Before building and installing the K Framework, the following prerequisites must first be installed.

The Short Version

Regardless of system, unless you cloned with --recursive you will first have to run git submodule update --init --recursive.

You then need the build and runtime dependencies. If you are on a Debian-based system (including Ubuntu) or MacOS with Homebrew installed, you can run ./install-build-deps and continue directly to the Build and Install Guide.

The Long Version

The following dependencies are needed either at build time or runtime:

  • bison
  • boost
  • cmake
  • flex
  • fmt
  • gcc
  • gmp
  • jdk (version 17 or greater)
  • libjemalloc
  • libsecp256k1
  • libyaml
  • llvm (We require version 15 or greater for clang, lld, and llvm-tools. On some distributions, the utilities below are also needed and packaged separately.)
  • make
  • maven
  • mpfr
  • pkg-config
  • python
  • stack
  • xxd
  • zlib
  • z3 (on some distributions libz3 is also needed and packaged separately) Note that you need version 4.12.1 of Z3, which may require you to build and install from source if your package manager does not supply it. Other versions are known to have bugs and performance regressions likely to cause issues in the K test suite.

Typically, these can all be installed from your package manager. On some system configurations, special installation steps or post-installation configuration steps are required. See the notes below.

Installation Notes

  1. Java Development Kit (required JDK 17 or higher)

    • Linux: Download from package manager (e.g. sudo apt-get install openjdk-17-jdk).

    • macOS/brew: Download from package manager (e.g. brew install java).

    To make sure that everything works you should be able to call java -version and javac -version from a terminal.

  2. LLVM

    • macOS/brew: Since LLVM is distributed as a keg-only package, we must explicitly make it available for command line usage. See the results of the brew info llvm command for more information on how to do this.

  3. Flex / Bison

    • macOS/brew: The versions of these packages supplied by the OS are too old, and are not compatible with the K build. You must ensure that the Homebrew-installed versions are first on your PATH when building K (i.e. which flex is not /usr/bin/flex).

  4. Apache Maven

    • Linux: Download from package manager (e.g. sudo apt-get install maven).

    • macOS/brew: Download it from a package manager or from http://maven.apache.org/download.cgi and follow the instructions on the webpage.

    Maven usually requires setting an environment variable JAVA_HOME pointing to the installation directory of the JDK (not to be mistaken with JRE).

    You can test if it works by calling mvn -version in a terminal. This will provide the information about the JDK Maven is using, in case it is the wrong one.

  5. Haskell Stack

    To install, go to https://docs.haskellstack.org/en/stable/README/ and follow the instructions. You may need to do stack upgrade to ensure the latest version of Haskell Stack.

Build and Install Guide

Building with Maven

Checkout the project source at your desired location and call mvn package from the main directory to build the distribution. For convenient usage, you can update your $PATH with <checkout-dir>/k-distribution/target/release/k/bin (strongly recommended, but optional).

You are also encouraged to set the environment variable MAVEN_OPTS to -XX:+TieredCompilation, which will significantly speed up the incremental build process.

Apple Silicon Support

K is fully tested and supported on ARM (M1/M2) family macOS machines. However, to work around an upstream bug in the Haskell / Stack ecosystem, care needs to be taken when initially building K from source. Before running any Maven commands, the Haskell Stack build needs to be configured without Homebrew's LLVM appearing on your $PATH:

First, run the following command from the K source root:

cd haskell-backend/src/main/native/haskell-backend && stack setup && cd -

Then, ensure that Homebrew-installed versions of llvm-config, flex and bison are on your PATH ahead of any macOS-supplied versions. direnv offers a convenient way to automate this. To do so:

brew install direnv
# Follow the instructions at https://direnv.net/docs/hook.html
# ...for example, if your shell is bash, run:
#   echo 'eval "$(direnv hook bash)"' >> ~/.bashrc
# then restart your shell.
cp macos-envrc .envrc
direnv allow
# You should see a message like:
#   direnv: loading .../k/.envrc
#   direnv: export ~PATH
# The llvm-config binary should also be on your PATH; check with:
which llvm-config

If you subsequently encounter errors when building the Haskell components of K (the Haskell backend and booster), try removing the entire Stack cache (~/.stack) and retrying the instructions above.

Building with Nix flakes (Recommended)

We now support building K using nix flakes. To set up nix flakes you will need to be on nix 2.4 or higher and follow the instructions here.

For example, if you are on a standard Linux distribution, such as Ubuntu, first install nix and then enable flakes by editing either ~/.config/nix/nix.conf or /etc/nix/nix.conf and adding:

experimental-features = nix-command flakes

This is needed to expose the Nix 2.0 CLI and flakes support that are hidden behind feature-flags.

By default, Nix will build the project and its transitive dependencies from source, which can take up to an hour. We recommend setting up the binary cache to speed up the build process significantly. You will also need to add the following sections to /etc/nix/nix.conf or, if you are a trusted user, ~/.config/nix/nix.conf (if you don't know what a "trusted user" is, you probably want to do the former):

trusted-public-keys = ... hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ=
substituters = ... https://cache.iog.io

i.e. if the file was originally

substituters = https://cache.nixos.org
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=

it will now read

substituters = https://cache.nixos.org https://cache.iog.io
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ=

To build the K Framework itself, run:

nix build .

This will build all of K and put a link to the resulting binaries in the result/ folder.

Note: Mac users, especially those running M1/M2 Macs may find nix segfaulting on occasion. If this happens, try running the nix command like this: GC_DONT_GC=1 nix build .

If you want to temporarily add the K binaries (such as kompile or kast) to the current shell, run

nix shell .

To run the integration tests:

nix build .#test

If you change any pom.xml, you must run

nix run .#update-maven

and commit the updated nix/mavenix.lock file.

IDE Setup

General

You should run K from the k-distribution project, because it is the only project to have the complete classpath and therefore all backends.

IntelliJ IDEA

IntelliJ IDEA comes with built-in maven integration. For more information, refer to the IntelliJ IDEA wiki

Running the Test Suite

To completely test the current version of the K framework, run mvn verify. This normally takes roughly 30 minutes on a fast machine. If you are interested only in running the unit tests and checkstyle goals, run mvn verify -DskipKTest to skip the lengthy ktest execution.

Building the Final Release Directory/Archives

Call mvn install in the base directory. This will attach an artifact to the local maven repository containing a zip and tar.gz of the distribution.

The functionality to create a tagged release is currently incomplete.

Compiling Definitions and Running Programs

Assuming k-distribution/target/release/k/bin is in your path, you can compile definitions using the kompile command. To execute a program you can use krun.

For running either program in the debugger, use the main class org.kframework.main.Main with an additional argument -kompile or -krun added before other command line arguments, and use the classpath from the k-distribution module.

Installing Python Support

Python tools for K can be found under runtimeverification/pyk.

Troubleshooting

Common build-time error messages:

  • Error: JAVA_HOME not found in your environment. Please set the JAVA_HOME variable in your environment to match the location of your Java installation.

    • Make sure JAVA_HOME points to the JDK and not the JRE directory.

  • [WARNING] Cannot get the branch information from the git repository: Detecting the current branch failed: 'git' is not recognized as an internal or external command, operable program or batch file.

    • git might not be installed on your system. Make sure that you can execute git from the command line.

  • 1) Error injecting constructor, java.lang.Error: Unresolved compilation problems: The import org.kframework.parser.outer.Outer cannot be resolved Outer cannot be resolved

    • You may run into this issue if target/generated-sources/javacc is not added to the build path of your IDE. Generally this is solved by regenerating your project / re-syncing it with the pom.xml.

  • [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (build-haskell) on project haskell-backend: An Ant BuildException has occured: exec returned: 1

    and scrolling up, you see an error message similar to:

    [exec] Installing GHC ... [exec] ghc-pkg: Couldn't open database $HOME/.stack/programs/x86_64-linux/ghc-tinfo6-8.10.1/lib/ghc-8.10.1/package.conf.d for modification: {handle: $HOME/.stack/programs/x86_64-linux/ghc-tinfo6-8.10.1/lib/ghc-8.10.1/package.conf.d/package.cache.lock}: hLock: invalid argument (Invalid argument)

    • If you are using a WSL version 1 environment, then you have encountered a known issue with the latest versions of GHC. In this case, please either:
      • upgrade to WSL version 2,
      • install a packaged release for your WSL version 1 distribution,
      • switch to a supported system configuration (e.g. Linux on a virtual machine), or
      • if you do not need the symbolic execution capabilities of the K Framework, disable them at build time (and remove the GHC dependency) by doing: mvn package -Dhaskell.backend.skip.

  • When building with nix and getting:

    • error: hash mismatch in fixed-output derivation '/nix/store/wjz7gjqs3cch9lgdjhs1fnb8wfl352vd-k-6.1.0-dirty-maven-deps.drv':
        specified: sha256-kLpjMj05uC94/5vGMwMlFzLKNFOKey◊Nvq/vmB6pHTAo=
            got:    sha256-fFlRqlLDZnVuoJniPvXjqdYEjnKxmFCEniavau/1gcQ=
error: 1 dependencies of derivation '/nix/store/79hazjbxp8829wpjvhh9c7kzc1m0ii22-k-6.1.0-dirty.drv' failed to build

      copy the got: hash (sha256-fFlRqlLDZnVuoJniPvXjqdYEjnKxmFCEniavau/1gcQ=) and replace it in flake.nix:

      k-framework = { haskell-backend-bins, llvm-kompile-libs }:
    prev.callPackage ./nix/k.nix {
        mvnHash = "sha256-fFlRqlLDZnVuoJniPvXjqdYEjnKxmFCEniavau/1gcQ=";
        ...

    • [ERROR] Failed to execute goal ... org.apache.maven.artifact.resolver.ArtifactNotFoundException: The following artifacts could not be resolved: org.scala-lang:scala-compiler:jar:2.12.18 ... Add "org.scala-lang:scala-compiler:2.12.18" (without the jar:) to manualMvnArtifacts in flake.nix

If something unexpected happens and the project fails to build, try mvn clean and rebuild the entire project. Generally speaking, however, the project should build incrementally without needing to be cleaned first.

If you are doing work with snapshot dependencies, you can update them to the latest version by running maven with the -U flag.

If you are configuring artifacts in a repository and need to purge the local repository's cache of artifacts, you can run mvn dependency:purge-local-repository.

If tests fail but you want to run the build anyway to see what happens, you can use mvn package -DskipTests.

If you still cannot build, please contact a K developer.