Skip to content
This repository has been archived by the owner on Oct 10, 2024. It is now read-only.

Commit

Permalink
Echidna tests and slither fixes (#543)
Browse files Browse the repository at this point in the history
* Fix existing tests and update README.md

* Parallelize CircleCI tests

* Ignore some slither detectors

* Ignore more slither detectors and fix tests

* Add slither configuration file

* Speed up mythril tests and remove unnecessary slither flags

* Reduce timeout and sign commit

* Change mythril timeout
  • Loading branch information
nostdm authored Mar 27, 2020
1 parent f8c93e3 commit 764c45a
Show file tree
Hide file tree
Showing 22 changed files with 241 additions and 442 deletions.
128 changes: 58 additions & 70 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,155 +46,143 @@ jobs:
working_directory: contracts
command: |
if [[ "${CIRCLE_NODE_INDEX}" == 0 ]]; then
myth analyze --solv=0.5.15 ./oracle.sol --execution-timeout=900
myth analyze --solv=0.5.15 ./oracle.sol --execution-timeout=800
fi
no_output_timeout: 20m
- run:
working_directory: contracts
command: |
if [[ "${CIRCLE_NODE_INDEX}" == 1 ]]; then
myth analyze --solv=0.5.15 ./wallet.sol --execution-timeout=900
myth analyze --solv=0.5.15 ./wallet.sol --execution-timeout=800
fi
no_output_timeout: 20m
- run:
working_directory: contracts
command: |
if [[ "${CIRCLE_NODE_INDEX}" == 2 ]]; then
myth analyze --solv=0.5.15 ./holder.sol --execution-timeout=900
myth analyze --solv=0.5.15 ./holder.sol --execution-timeout=800
fi
no_output_timeout: 20m
- run:
working_directory: contracts
command: |
if [[ "${CIRCLE_NODE_INDEX}" == 3 ]]; then
myth analyze --solv=0.5.15 ./licence.sol --execution-timeout=900
myth analyze --solv=0.5.15 ./licence.sol --execution-timeout=800
fi
no_output_timeout: 20m
- run:
working_directory: contracts
command: |
if [[ "${CIRCLE_NODE_INDEX}" == 4 ]]; then
myth analyze --solv=0.5.15 ./tokenWhitelist.sol --execution-timeout=900
myth analyze --solv=0.5.15 ./tokenWhitelist.sol --execution-timeout=800
fi
no_output_timeout: 20m
- run:
working_directory: contracts
command: |
if [[ "${CIRCLE_NODE_INDEX}" == 5 ]]; then
myth analyze --solv=0.5.15 ./walletDeployer.sol --execution-timeout=900
myth analyze --solv=0.5.15 ./walletDeployer.sol --execution-timeout=800
fi
no_output_timeout: 20m

slither:
docker:
- image: trailofbits/eth-security-toolbox:latest
working_directory: /tmp/contracts
parallelism: 6
steps:
- run: sudo -n apt-get update && sudo -n apt-get install -y openssh-client
- checkout
- run:
command: solc-select 0.5.15
- run: solc-select 0.5.15
- run:
working_directory: slither
command: |
slither ../contracts/wallet.sol ||
echo "export FAILED=true" >> $BASH_ENV
if [[ "${CIRCLE_NODE_INDEX}" == 0 ]]; then
slither ../contracts/wallet.sol
fi
no_output_timeout: 5m
- run:
working_directory: slither
command: |
slither ../contracts/oracle.sol ||
echo "export FAILED=true" >> $BASH_ENV
if [[ "${CIRCLE_NODE_INDEX}" == 1 ]]; then
slither ../contracts/oracle.sol
fi
no_output_timeout: 5m
- run:
working_directory: slither
command: |
slither ../contracts/licence.sol ||
echo "export FAILED=true" >> $BASH_ENV
if [[ "${CIRCLE_NODE_INDEX}" == 2 ]]; then
slither ../contracts/licence.sol
fi
no_output_timeout: 5m
- run:
working_directory: slither
command: |
slither ../contracts/holder.sol ||
echo "export FAILED=true" >> $BASH_ENV
if [[ "${CIRCLE_NODE_INDEX}" == 3 ]]; then
slither ../contracts/holder.sol
fi
no_output_timeout: 5m
- run:
working_directory: slither
command: |
slither ../contracts/tokenWhitelist.sol ||
echo "export FAILED=true" >> $BASH_ENV
if [[ "${CIRCLE_NODE_INDEX}" == 4 ]]; then
slither ../contracts/tokenWhitelist.sol
fi
no_output_timeout: 5m
- run:
working_directory: slither
command: |
slither ../contracts/walletDeployer.sol ||
echo "export FAILED=true" >> $BASH_ENV
if [[ "${CIRCLE_NODE_INDEX}" == 5 ]]; then
slither ../contracts/walletDeployer.sol
fi
no_output_timeout: 5m
- run:
command: test ! $FAILED

echidna:
docker:
- image: trailofbits/eth-security-toolbox:latest
working_directory: /tmp/contracts
parallelism: 6
steps:
- run: sudo -n apt-get update && sudo -n apt-get install -y openssh-client
- checkout
- run:
command: solc-select 0.5.15
- run:
command: |
slither-flat --convert-external contracts/wallet.sol &&
echidna-test echidna/addressWhitelist.sol --config echidna/addressWhitelist.yaml TEST ||
echo "export FAILED=true" >> $BASH_ENV
no_output_timeout: 20m
- run:
command: |
slither-flat --convert-external contracts/controller.sol &&
echidna-test echidna/controller.sol --config echidna/controller.yaml TEST ||
echo "export FAILED=true" >> $BASH_ENV
no_output_timeout: 20m
- run:
command: |
echidna-test echidna/date.sol --config echidna/date.yaml TEST ||
echo "export FAILED=true" >> $BASH_ENV
no_output_timeout: 20m
- run:
command: |
echidna-test echidna/ecRecover.sol --config echidna/ecRecover.yaml TEST ||
echo "export FAILED=true" >> $BASH_ENV
no_output_timeout: 20m
- run: solc-select 0.5.15
- run: find contracts -maxdepth 1 -type f -name '*.sol' -exec slither-flat --convert-external {} \;
- run:
command: |
echidna-test echidna/ecRecover.sol --config echidna/ecRecover.yaml TEST ||
echo "export FAILED=true" >> $BASH_ENV
no_output_timeout: 20m
if [[ "${CIRCLE_NODE_INDEX}" == 0 ]]; then
echidna-test echidna/addressWhitelist.sol --config=echidna/addressWhitelist.yaml --contract=TEST
fi
no_output_timeout: 15m
- run:
command: |
echidna-test echidna/gasTopUpLimit.sol --config echidna/gasTopUpLimit.yaml TEST ||
echo "export FAILED=true" >> $BASH_ENV
no_output_timeout: 20m
if [[ "${CIRCLE_NODE_INDEX}" == 1 ]]; then
echidna-test echidna/controller.sol --config=echidna/controller.yaml --contract=TEST
fi
no_output_timeout: 15m
- run:
command: |
echidna-test echidna/ownable.sol --config echidna/ownable.yaml TEST ||
echo "export FAILED=true" >> $BASH_ENV
no_output_timeout: 20m
if [[ "${CIRCLE_NODE_INDEX}" == 2 ]]; then
echidna-test echidna/date.sol --config=echidna/date.yaml --contract=TEST
fi
no_output_timeout: 15m
- run:
command: |
echidna-test echidna/parseJson.sol --config echidna/parseJson.yaml TEST ||
echo "export FAILED=true" >> $BASH_ENV
no_output_timeout: 20m
if [[ "${CIRCLE_NODE_INDEX}" == 3 ]]; then
echidna-test echidna/gasTopUpLimit.sol --config=echidna/gasTopUpLimit.yaml --contract=TEST
fi
no_output_timeout: 15m
- run:
command: |
echidna-test echidna/spendLimit.sol --config echidna/spendLimit.yaml TEST ||
echo "export FAILED=true" >> $BASH_ENV
no_output_timeout: 20m
if [[ "${CIRCLE_NODE_INDEX}" == 4 ]]; then
echidna-test echidna/ownable.sol --config=echidna/ownable.yaml --contract=TEST
fi
no_output_timeout: 15m
- run:
command: |
echidna-test echidna/stringProps.sol --config echidna/stringProps.yaml TEST ||
echo "export FAILED=true" >> $BASH_ENV
no_output_timeout: 20m
- run:
command: test ! $FAILED
if [[ "${CIRCLE_NODE_INDEX}" == 5 ]]; then
echidna-test echidna/spendLimit.sol --config=echidna/spendLimit.yaml --contract=TEST
fi
no_output_timeout: 15m

workflows:
version: 2
Expand All @@ -218,7 +206,7 @@ workflows:
filters:
tags:
ignore: ""
# - echidna:
# filters:
# tags:
# ignore: ""
- echidna:
filters:
tags:
ignore: ""
64 changes: 39 additions & 25 deletions echidna/README.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,48 @@
# Echidna tests for TokenCard contracts
# Properties

This directory contains some properties for testing TokenCard contracts where three users are defined:
This directory contains some properties for testing Solidity contracts where multiple users are defined:

* `echidna_owner`: the owner of the contract only used to execute the contract constructor
* `echidna_attack`: an unprivileged user, used to execute the randomly generated transactions
* `echidna_user`: another unprivileged user, also used to execute randomly generated transactions in cases where multiple users are necessary
- `echidna_deployer`: Deployer address used to execute the contract constructor.

We test a variety of properties, such as:
* Whitelist addition and removal operations properly modify the whitelist
* The owner of a contract cannot be changed.
* Spending limits are properly bounded by the spend limit values
* Spending limits do in fact reset daily
* Differential testing of the different interger parsing functions produce identical results
- `echidna_owner`: Address of the user that owns an ownable contract.

- `echidna_controller`: Controller address assigned to a controllable contract.

- `echidna_attacker`: An unprivileged user, used to execute the randomly generated transactions.

# Configuration

Individual tests can be configured using yaml configuration files.

A full list of test configuration options can be found here: [default.yaml](https://github.com/crytic/echidna/blob/master/examples/solidity/basic/default.yaml)

# Instructions

1. Uncompress this file in the root of [TokenCard's contract repository](https://github.com/tokencard/contracts). We tested this example with version [2.0.0](https://github.com/tokencard/contracts/commit/b99b7d1670f9ad7b90335e8391fe63fd7e20de9b).
2. Install the latest revision of [Echidna](https://github.com/crytic/echidna/#installation). Note that `ecRecover.sol` requires the `dev-hevm-0.29` branch of Echidna to run.
3. Execute `echidna-test` with one of the `*.sol` files and its corresponding `*.yaml` config file. For example, an invocation with `spendLimit.sol` would look like:
Normally the echidna tests run in CircleCI but they can also be invoked locally for development purposes.

Run trail of bits security toolbox and mount the contracts directory, you need to make sure that the path is an absolute path:

docker run -v $GOPATH/src/github.com/tokencard/contracts:/contracts -it trailofbits/eth-security-toolbox:latest

Run `echidna-test` command on each test contract:

echidna-test echidna/spendLimit.sol --config=echidna/spendLimit.yaml --contract=TEST

Analyzing contract: spendLimit.sol:TEST
echidna_fixed_spendLimitValue: passed!
echidna_bounded_spendLimitAvailable: passed!
echidna_fixed_owner: passed!
echidna_daily_spendLimitAvailable: passed!

Unique instructions: 900
Unique codehashes: 1

# Tools

Sometimes it's necessary to convert external functions in a contract to public functions so that they can be called inside the derived TEST contract.

```
$ echidna-test spendLimit.sol --config spendLimit.yaml TEST
...
For this it's advisable to use the `slither-flat` command like so:

Analyzing contract: spendLimit.sol:TEST
echidna_fixed_spendLimitValue: passed! πŸŽ‰
echidna_bounded_spendLimitAvailable: passed! πŸŽ‰
echidna_fixed_owner: passed! πŸŽ‰
echidna_daily_spendLimitAvailable: passed! πŸŽ‰
slither-flat --convert-external contracts/wallet.sol

Unique instructions: 900
Unique codehashes: 1
```
This will create a new flattened contract at `critic-export/flattening/Wallet.sol` which can then be imported in the test file as `import "crytic-export/flattening/Wallet.sol";`.
Loading

0 comments on commit 764c45a

Please sign in to comment.