Skip to content

Commit

Permalink
Clean up Docker containers (#706)
Browse files Browse the repository at this point in the history
* Dockerfile: make image slimmer, based on distroless

* GitHub Actions: configure docker buildx

* GitHub Actions: move to ghcr.io

* Move dockerfile, add solc-install wrapper

* Add solc-select to python environment

* Disable wheel caching

* Add debian-based image

* Build and tag both debian and distroless images

* Reorganize tags, use build and push action

* Add NVM variant

* Indicate Dockerfile path correctly

* Unify Debian variants

* Switch to Docker metadata action

* Improve Docker section on README

* Enable caching of layers

* Update action versions

* Move back to Ubuntu

To keep the same distro we were using previously

* Disable building distroless

* Publish the images under the previous names

* Fix naming and publishing

* Use ubuntu focal

Try fixing missing __xmknod symbol with ubuntu jammy

* Make apt-get installs non-interactive

* Remove 'ubuntu' ghcr image name

* Add setuptools, distutils and pip

Otherwise the final environment is slightly broken if someone tries
to install extra packages

* Add UTF-8 locale

Echidna prints emojis and will throw an error if the locale is not
set correctly.

* README: fix some markdownlint warnings

* README: adjust Docker documentation
  • Loading branch information
elopez authored Sep 9, 2022
1 parent aab102b commit 1dc209b
Show file tree
Hide file tree
Showing 5 changed files with 197 additions and 84 deletions.
108 changes: 65 additions & 43 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,64 +5,86 @@ on:
branches:
- master
- github-docker
- docker-slim
tags:
- '*'

env:
WORKFLOW_BUILD_DISTROLESS: false

jobs:
docker:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v3

- name: GitHub Login
uses: azure/docker-login@v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
id: buildx
with:
login-server: docker.pkg.github.com
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
install: true

- name: Set Docker Package and Version
run: |
set +e
_=$(echo "$GITHUB_REF" | grep "^refs/heads/")
if [ $? -eq 0 ]; then
# branch
if [ "$GITHUB_REF" = "refs/heads/master" ]; then
VER=latest
PKG=echidna
else
VER=testing
PKG=testing
fi
fi
_=$(echo "$GITHUB_REF" | grep "^refs/tags/")
if [ $? -eq 0 ]; then
# tag
# refs/tags/v1.X => v1.X
VER=$(echo "$GITHUB_REF" | sed -e 's/.*\///')
PKG=echidna
fi
set -e
echo "PKG=$PKG" >> $GITHUB_ENV
echo "VER=$VER" >> $GITHUB_ENV
- name: Set Docker metadata (Ubuntu & NVM variant)
id: meta-ubuntu
uses: docker/metadata-action@v4
with:
images: |
ghcr.io/${{ github.repository }}/echidna
name=trailofbits/echidna,enable=${{ github.repository == 'crytic/echidna' }}
tags: |
type=ref,event=tag
type=ref,event=branch,prefix=testing-
type=edge
- name: Docker Build
run: docker build -t $PKG:$VER .
- name: Set Docker metadata (Distroless variant)
id: meta-distroless
uses: docker/metadata-action@v4
if: ${{ env.WORKFLOW_BUILD_DISTROLESS == true }}
with:
images: ghcr.io/${{ github.repository }}/distroless
tags: |
type=ref,event=tag
type=ref,event=branch,prefix=testing-
type=edge
- name: Push to GitHub Packages
run: |
docker tag $PKG:$VER docker.pkg.github.com/$GITHUB_REPOSITORY/$PKG:$VER
docker push docker.pkg.github.com/$GITHUB_REPOSITORY/$PKG:$VER
- name: GitHub Container Registry Login
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: DockerHub Login
uses: azure/docker-login@v1
- name: Docker Hub Login
uses: docker/login-action@v2
if: github.repository == 'crytic/echidna'
with:
login-server: registry.hub.docker.com
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_ACCESS_TOKEN }}

- name: Push to DockerHub
run: |
docker tag $PKG:$VER registry.hub.docker.com/trailofbits/echidna:$VER
docker push registry.hub.docker.com/trailofbits/echidna:$VER
- name: Docker Build and Push (Ubuntu & NVM variant)
uses: docker/build-push-action@v3
with:
platforms: linux/amd64
target: final-ubuntu
file: docker/Dockerfile
pull: true
push: true
tags: ${{ steps.meta-ubuntu.outputs.tags }}
labels: ${{ steps.meta-ubuntu.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max

- name: Docker Build and Push (Distroless variant)
uses: docker/build-push-action@v3
if: ${{ env.WORKFLOW_BUILD_DISTROLESS == true }}
with:
platforms: linux/amd64
target: final-distroless
file: docker/Dockerfile
pull: true
push: true
tags: ${{ steps.meta-distroless.outputs.tags }}
labels: ${{ steps.meta-distroless.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
22 changes: 0 additions & 22 deletions Dockerfile

This file was deleted.

74 changes: 55 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,13 @@ function echidna_check_balance() public returns (bool) {

To check these invariants, run:

```
```sh
$ echidna-test myContract.sol
```

An example contract with tests can be found [tests/solidity/basic/flags.sol](tests/solidity/basic/flags.sol). To run it, you should execute:
```

```sh
$ echidna-test tests/solidity/basic/flags.sol
```

Expand All @@ -57,7 +58,7 @@ After finishing a campaign, Echidna can save a coverage maximizing **corpus** in

If you run `tests/solidity/basic/flags.sol` example, Echidna will save a few files serialized transactions in the `coverage` directory and a `covered.$(date +%s).txt` file with the following lines:

```
```text
*r | function set0(int val) public returns (bool){
* | if (val % 100 == 0)
* | flag0 = false;
Expand All @@ -70,10 +71,11 @@ If you run `tests/solidity/basic/flags.sol` example, Echidna will save a few fil
```

Our tool signals each execution trace in the corpus with the following "line marker":
- `*` if an execution ended with a STOP
- `r` if an execution ended with a REVERT
- `o` if an execution ended with an out-of-gas error
- `e` if an execution ended with any other error (zero division, assertion failure, etc)

* `*` if an execution ended with a STOP
* `r` if an execution ended with a REVERT
* `o` if an execution ended with an out-of-gas error
* `e` if an execution ended with any other error (zero division, assertion failure, etc)

### Support for smart contract build systems

Expand All @@ -97,7 +99,7 @@ usage instructions and examples.
Echidna's CLI can be used to choose the contract to test and load a
configuration file.

```
```sh
$ echidna-test contract.sol --contract TEST --config config.yaml
```

Expand Down Expand Up @@ -178,17 +180,48 @@ If you want to quickly test Echidna in Linux or MacOS, we provide statically lin

### Docker container

If you prefer to use a pre-built Docker container, log into Github on your local `docker` client and check out our [docker package](https://github.com/crytic/echidna/packages/136575), which are also auto-built via Github Actions.
Otherwise, if you want to install the latest released version of Echidna, we recommend using docker:

```
$ docker build -t echidna .
If you prefer to use a pre-built Docker container, check out our [docker
package](https://github.com/orgs/crytic/packages?repo_name=echidna), which is
auto-built via Github Actions. The `echidna` container is based on
`ubuntu:focal` and it is meant to be a small yet flexible enough image to use
Echidna on. It provides a pre-built version of `echidna-test`, as well as
`slither`, `crytic-compile`, `solc-select` and `nvm` under 200 MB.

Note that the container images currently only build on x86 systems. Running them
on ARM devices, such as Mac M1 systems, is not recommended due to the performance
loss incurred by the CPU emulation.

Different tags are available for the Docker container image:

| Tag | Build in tag
|---------------|-------------
| `vx.y.z` | Build corresponding to release `vx.y.z`
| `latest` | Latest Echidna tagged release.
| `edge` | Most recent commit on the default branch.
| `testing-foo` | Testing build based on the `foo` branch.

To run the container with the latest Echidna version interactively, you can use
something like the following command. It will map the current directory as
`/src` inside the container, and give you a shell where you can use
`echidna-test`:

```sh
$ docker run --rm -it -v `pwd`:/src ghcr.io/crytic/echidna/echidna
```

Then, run it via:
Otherwise, if you want to locally build the latest version of Echidna, we
recommend using Docker. From within a clone of this repository, run the
following command to build the Docker container image:

```sh
$ docker build -t echidna -f docker/Dockerfile --target final-ubuntu .
```
$ docker run -it -v `pwd`:/src echidna echidna-test /src/tests/solidity/basic/flags.sol

Then, you can run the `echidna` image locally. For example, to install solc
0.5.7 and check `tests/solidity/basic/flags.sol`, you can run:

```sh
$ docker run -it -v `pwd`:/src echidna bash -c "solc-select install 0.5.7 && solc-select use 0.5.7 && echidna-test /src/tests/solidity/basic/flags.sol"
```

### Building using Stack
Expand All @@ -202,14 +235,15 @@ If you're getting errors building related to linking, try tinkering with `--extr
### Building using Nix (works natively on Apple M1 systems)

[Nix users](https://nixos.org/download.html) can install the lastest Echidna with:
```

```sh
$ nix-env -i -f https://github.com/crytic/echidna/tarball/master
```

To build a standalone release for non-Nix macOS systems, the following will
bundle Echidna and all linked dylibs in a tarball:

```
```sh
$ nix-build macos-release.nix
$ ll result/
bin echidna-1.7.3-aarch64-darwin.tar.gz
Expand All @@ -218,15 +252,17 @@ bin echidna-1.7.3-aarch64-darwin.tar.gz
It is possible to develop Echidna with Cabal inside `nix-shell`. Nix will automatically
install all the dependencies required for development including `crytic-compile` and `solc`.
A quick way to get GHCi with Echidna ready for work:
```

```sh
$ git clone https://github.com/crytic/echidna
$ cd echidna
$ nix-shell
[nix-shell]$ cabal new-repl
```

Running the test suite:
```

```sh
nix-shell --run 'cabal test'
```

Expand Down
55 changes: 55 additions & 0 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
FROM ubuntu:focal AS builder-echidna
ENV LD_LIBRARY_PATH=/usr/local/lib PREFIX=/usr/local HOST_OS=Linux
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-suggests --no-install-recommends \
cmake \
curl \
git \
libbz2-dev \
libgmp-dev \
libreadline-dev \
libsecp256k1-dev \
libssl-dev \
software-properties-common \
sudo
RUN curl -sSL https://get.haskellstack.org/ | sh
COPY . /echidna/
WORKDIR /echidna
RUN .github/scripts/install-libff.sh
RUN stack upgrade && stack setup && stack install --extra-include-dirs=/usr/local/include --extra-lib-dirs=/usr/local/lib


FROM ubuntu:focal AS builder-python3
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-suggests --no-install-recommends \
gcc \
python3.8-dev \
python3.8-venv
ENV PIP_DISABLE_PIP_VERSION_CHECK=1
ENV PIP_NO_CACHE_DIR=1
RUN python3 -m venv /venv && /venv/bin/pip3 install --no-cache --upgrade setuptools pip
RUN /venv/bin/pip3 install --no-cache slither-analyzer solc-select


FROM gcr.io/distroless/python3-debian11:nonroot AS final-distroless
COPY --from=builder-echidna /root/.local/bin/echidna-test /usr/local/bin/echidna-test
COPY --from=builder-python3 /venv /venv
COPY docker/solc-install.py /usr/local/bin/solc-install
ENV PATH="$PATH:/venv/bin"
ENTRYPOINT [ "/usr/local/bin/solc-install", "/usr/local/bin/echidna-test" ]


FROM ubuntu:focal AS final-ubuntu
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-suggests --no-install-recommends \
ca-certificates \
curl \
python3 \
python3-distutils \
&& \
rm -rf /var/lib/apt/lists/*
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
COPY --from=builder-echidna /root/.local/bin/echidna-test /usr/local/bin/echidna-test
COPY --from=builder-python3 /venv /venv
ENV LANG="C.UTF-8"
ENV PATH="$PATH:/venv/bin"
22 changes: 22 additions & 0 deletions docker/solc-install.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/usr/bin/python3
import os
import shutil
import subprocess
import sys

## solc-install: simple wrapper script to invoke solc-select install when required
##
## This script will observe the SOLC_VERSION variable. If it is set, it will install
## and globally select said solc version.

if len(sys.argv) < 2:
print(f"Usage: {sys.argv[0]} other-program [args..]")
sys.exit(1)

solc_version = os.getenv('SOLC_VERSION', None)
if solc_version:
silent = os.getenv("SOLC_SELECT_SILENT", "1") == "1" and subprocess.DEVNULL or None
subprocess.run(['solc-select', 'install', solc_version], stderr=silent, stdout=silent)
subprocess.run(['solc-select', 'use', solc_version], stderr=silent, stdout=silent)

os.execv(shutil.which(sys.argv[1]), sys.argv[1:])

0 comments on commit 1dc209b

Please sign in to comment.