Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(test): fuzzed and symbolic tests #146

Merged
merged 43 commits into from
Aug 2, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
21bc497
Merge commit 'ce1b925d296ac1bb5dbaf5c688c8592d43f64e36' into feat/fuz…
simon-something Jul 9, 2024
5b1182b
feat: echidna and halmos bmath bnum protocol
simon-something Jul 9, 2024
bec1de2
fix: remove tests halmos cannot handle
simon-something Jul 9, 2024
1d0d05d
fix: setup cow pool
simon-something Jul 9, 2024
013f72f
chore(wip): echidna
simon-something Jul 9, 2024
388c657
feat: clamp util
simon-something Jul 11, 2024
caf3c62
feat(echidna): prop and tob erc20
simon-something Jul 12, 2024
b01b4e6
test(echidna): protocol prop (part)
simon-something Jul 14, 2024
3753371
test(echidna): swap/join prop (part)
simon-something Jul 15, 2024
d12abd4
test(echidna): all properties
simon-something Jul 15, 2024
8cde0c0
test(halmos): part
simon-something Jul 15, 2024
0d0ca16
feat: summary prop md and last tests
simon-something Jul 16, 2024
87a9acb
chore: reorg
simon-something Jul 16, 2024
e946f09
feat: more bnum pain
simon-something Jul 16, 2024
59c2686
chore: format
simon-something Jul 20, 2024
462c38e
chore: format
simon-something Jul 20, 2024
68a6b59
chore: typo
simon-something Jul 20, 2024
fdd4617
chore: typo
simon-something Jul 20, 2024
cad698a
test(echidna): more bnum tests
simon-something Jul 22, 2024
c143808
chore: unused import
simon-something Jul 22, 2024
a65a9c0
chore: more assert in protocol
simon-something Jul 22, 2024
350c39d
feat: evm version
simon-something Jul 23, 2024
c60f5fa
chore: summary
simon-something Jul 23, 2024
c5c263f
fix: direct transfer then swapExactOut case
simon-something Jul 23, 2024
09d332d
chore: fmt
simon-something Jul 23, 2024
888ebe9
feat: shanghai in summary
simon-something Jul 23, 2024
0c0e1c4
fix: typo
simon-something Jul 26, 2024
00cd7d1
chore: typo
simon-something Jul 26, 2024
ea280d9
chore: revert consistency
simon-something Jul 26, 2024
5555afc
chore: summary fmt
simon-something Jul 26, 2024
637aa83
chore: fmt
simon-something Jul 26, 2024
42d2a04
Fix/fuzz tests improvements (#178)
0xteddybear Jul 29, 2024
302463c
chore: merge dev
wei3erHase Jul 29, 2024
23a8264
chore: update forge snapshots
wei3erHase Jul 29, 2024
b3f2cf6
chore: merge
wei3erHase Jul 29, 2024
5abef51
chore: update gas snapshots
wei3erHase Jul 29, 2024
a793e9d
chore: updating Properties document after merge
wei3erHase Jul 31, 2024
61a0930
Merge pull request #185 from defi-wonderland/chore/merge-dev-into-fuzz
wei3erHase Jul 31, 2024
df49eb9
dev: improving bmath fuzz test (#184)
wei3erHase Aug 1, 2024
7bbb6cc
Merge branch 'dev' into feat/fuzz-symb-tests
wei3erHase Aug 1, 2024
9d976ff
docs: update test summary (#189)
0xteddybear Aug 1, 2024
b101648
Merge branch 'dev' into feat/fuzz-symb-tests
wei3erHase Aug 1, 2024
cd1ef62
chore: fuzz sym fixes (#190)
0xteddybear Aug 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .solhintignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
test/smock/*
test/manual-smock/*
test/invariant/*
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,8 @@
"@defi-wonderland/natspec-smells": "1.1.3",
"@defi-wonderland/smock-foundry": "1.5.0",
"forge-gas-snapshot": "github:marktoda/forge-gas-snapshot#9161f7c",
"forge-std": "github:foundry-rs/forge-std#5475f85",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will change the deployed build...

"forge-std": "github:foundry-rs/forge-std#1.8.2",
"halmos-cheatcodes": "github:a16z/halmos-cheatcodes#c0d8655",
"husky": ">=8",
"lint-staged": ">=10",
"solhint-community": "4.0.0",
Expand Down
1 change: 1 addition & 0 deletions remappings.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ solmate/=node_modules/solmate/src
@cowprotocol/=node_modules/@cowprotocol/contracts/src/contracts
cowprotocol/=node_modules/@cowprotocol/contracts/src/
@composable-cow/=node_modules/composable-cow/
halmos-cheatcodes=node_modules/halmos-cheatcodes

contracts/=src/contracts
interfaces/=src/interfaces
67 changes: 67 additions & 0 deletions test/SUMMARY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Outline

There are 7 files to cover (B(cow)Const are contracts containing public protocol-wide constants):
BCoWFactory: deployer for cow pools
BCoWPool: adding signature validation to BPool
BFactory: deployer, approx 40sloc
BMath: contract wrapping math logic
BNum: contract wrapping arithmetic op
BPool: the main contract, roughly 400sloc
simon-something marked this conversation as resolved.
Show resolved Hide resolved
BToken: extends erc20

## Interdependencies
BCoWFactory deploys a bcowpool
BCoWPool is a bpool which adds signature validation
Factory deploys a pool and can "collect" from the pool
Pool inherit btoken (which represents a LP) and bmath
Bmath uses bnum

## Approach

Echidna should be prioritized, then halmos should be particularly easy especially for the math libs, for which the implementations will be pretty similar.

Then slither-mutate on the whole test base

Setup for protocol-wide *looks* pretty simple (using the factory) - tbc

## Notes
The bmath corresponding equations are:

`Spot price:`
$\text{spotPrice} = \frac{\text{tokenBalanceIn}/\text{tokenWeightIn}}{\text{tokenBalanceOut}/\text{tokenWeightOut}} \cdot \frac{1}{1 - \text{swapFee}}$


`Out given in:`
$\text{tokenAmountOut} = \text{tokenBalanceOut} \cdot \left( 1 - \left( \frac{\text{tokenBalanceIn}}{\text{tokenBalanceIn} + \left( \text{tokenAmountIn} \cdot \left(1 - \text{swapFee}\right)\right)} \right)^{\frac{\text{tokenWeightIn}}{\text{tokenWeightOut}}} \right)$

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this expression doesn't seem to be renderable

Also, perhaps expressions as blocks would be easier to read? text in inline expressions gets real small

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

true true



`In given out:`
$\text{tokenAmountIn} = \frac{\text{tokenBalanceIn} \cdot \left( \frac{\text{tokenBalanceOut}}{\text{tokenBalanceOut} - \text{tokenAmountOut}} \right)^{\frac{\text{tokenWeightOut}}{\text{tokenWeightIn}}} - 1}{1 - \text{swapFee}}$


`Pool out given single in`
$\text{poolAmountOut} = \left(\frac{\text{tokenAmountIn} \cdot \left(1 - \left(1 - \frac{\text{tokenWeightIn}}{\text{totalWeight}}\right) \cdot \text{swapFee}\right) + \text{tokenBalanceIn}}{\text{tokenBalanceIn}}\right)^{\frac{\text{tokenWeightIn}}{\text{totalWeight}}} \cdot \text{poolSupply} - \text{poolSupply}$


`Single in given pool out`
$\text{tokenAmountIn} = \frac{\left(\frac{\text{poolSupply} + \text{poolAmountOut}}{\text{poolSupply}}\right)^{\frac{1}{\frac{\text{weightIn}}{\text{totalWeight}}}} \cdot \text{balanceIn} - \text{balanceIn}}{\left(1 - \frac{\text{weightIn}}{\text{totalWeight}}\right) \cdot \text{swapFee}}$


`Single out given pool in`
$\text{tokenAmountOut} = \left( \text{tokenBalanceOut} - \left( \frac{\text{poolSupply} - \left(\text{poolAmountIn} \cdot \left(1 - \text{exitFee}\right)\right)}{\text{poolSupply}} \right)^{\frac{1}{\frac{\text{tokenWeightOut}}{\text{totalWeight}}}} \cdot \text{tokenBalanceOut} \right) \cdot \left(1 - \left(1 - \frac{\text{tokenWeightOut}}{\text{totalWeight}}\right) \cdot \text{swapFee}\right)$


`Pool in given single out`
$\text{poolAmountIn} = \frac{\text{poolSupply} - \left( \frac{\text{tokenBalanceOut} - \frac{\text{tokenAmountOut}}{1 - \left(1 - \frac{\text{tokenWeightOut}}{\text{totalWeight}}\right) \cdot \text{swapFee}}}{\text{tokenBalanceOut}} \right)^{\frac{\text{tokenWeightOut}}{\text{totalWeight}}} \cdot \text{poolSupply}}{1 - \text{exitFee}}$


BNum bpow is based on exponentiation by squaring and hold true because (see dapphub dsmath): https://github.com/dapphub/ds-math/blob/e70a364787804c1ded9801ed6c27b440a86ebd32/src/math.sol#L62
```
// If n is even, then x^n = (x^2)^(n/2).
// If n is odd, then x^n = x * x^(n-1),
// and applying the equation for even x gives
// x^n = x * (x^2)^((n-1) / 2).
//
// Also, EVM division is flooring and
// floor[(n-1) / 2] = floor[n / 2].
```
60 changes: 60 additions & 0 deletions test/invariant/AdvancedTestsUtils.sol
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
// SPDX-License-Identifier: UNLICENSED
pragma solidity 0.8.25;

import {MockERC20} from 'forge-std/mocks/MockERC20.sol';
import {SymTest} from 'halmos-cheatcodes/src/SymTest.sol';

interface IHevm {
function prank(address) external;
}

contract FuzzERC20 is MockERC20 {
function mint(address _to, uint256 _amount) public {
_mint(_to, _amount);
}

function burn(address _from, uint256 _amount) public {
_burn(_from, _amount);
}
}

contract AgentsHandler {
uint256 internal agentsIndex;
address[] internal agents;

address internal currentCaller;

modifier AgentOrDeployer() {
uint256 _currentAgentIndex = agentsIndex;
currentCaller = _currentAgentIndex == 0 ? address(this) : agents[agentsIndex];
_;
}

constructor(uint256 _numAgents) {
for (uint256 i = 0; i < _numAgents; i++) {
agents.push(address(bytes20(keccak256(abi.encodePacked(i)))));
}
}

function nextAgent() public {
agentsIndex = (agentsIndex + 1) % agents.length;
}

function getCurrentAgent() public view returns (address) {
return agents[agentsIndex];
}
}

contract EchidnaTest is AgentsHandler {
event AssertionFailed();

IHevm hevm = IHevm(0x7109709ECfa91a80626fF3989D68f67F5b1DD12D);

constructor() AgentsHandler(5) {}

function clamp(uint256 _value, uint256 _min, uint256 _max) internal returns (uint256) {
return _min + (_value % (_max - _min));
}
}

contract HalmosTest is SymTest {}
99 changes: 99 additions & 0 deletions test/invariant/PROPERTIES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
| Properties | Type | Id | Halmos | Echidna |
| ------------------------------------------------------------------------------------------- | ------------------- | --- | ------ | ------- |
| BFactory should always be able to deploy new pools | Unit | 1 | [ ] | [ ] |
| BFactory's blab should always be modifiable by the current blabs | Unit | 2 | [ ] | [ ] |
| BFactory should always be able to transfer the BToken to the blab, if called by it | Unit | 3 | [ ] | [ ] |
| the amount received can never be less than min amount out | Unit | 4 | [ ] | [ ] |
| the amount spent can never be greater than max amount in | Unit | 5 | [ ] | [ ] |
| swap fee can only be 0 (cow pool) | Valid state | 6 | [ ] | [ ] |
| total weight can be up to 50e18 | Variable transition | 7 | [ ] | [ ] |
| BToken increaseApproval should increase the approval of the address by the amount | Variable transition | 8 | [ ] | [ ] |
| BToken decreaseApproval should decrease the approval to max(old-amount, 0) | Variable transition | 9 | [ ] | [ ] |
| a pool can either be finalized or not finalized | Valid state | 10 | [ ] | [ ] |
| a finalized pool cannot switch back to non-finalized | State transition | 11 | [ ] | [ ] |
| a non-finalized pool can only be finalized when the controller calls finalize() | State transition | 12 | [ ] | [ ] |
| an exact amount in should always earn the amount out calculated in bmath | High level | 13 | [ ] | [ ] |
| an exact amount out is earned only if the amount in calculated in bmath is transfered | High level | 14 | [ ] | [ ] |
| there can't be any amount out for a 0 amount in | High level | 15 | [ ] | [ ] |
| the pool btoken can only be minted/burned in the join and exit operations | High level | 16 | [ ] | [ ] |
| a direct token transfer can never reduce the underlying amount of a given token per BPT | High level | 17 | [ ] | [ ] |
| the amount of underlying token when exiting should always be the amount calculated in bmath | High level | 18 | [ ] | [ ] |
| a swap can only happen when the pool is finalized | High level | 19 | [ ] | [ ] |
| bounding and unbounding token can only be done on a non-finalized pool, by the controller | High level | 20 | [ ] | [ ] |
| there always should be between MIN_BOUND_TOKENS and MAX_BOUND_TOKENS bound in a pool | High level | 21 | [ ] | [ ] |
| only the settler can commit a hash | High level | 22 | [ ] | [ ] |
| when a hash has been commited, only this order can be settled | High level | 23 | [ ] | [ ] |
| BToken should not break the ToB ERC20 properties* | High level | 24 | [ ] | [ ] |

* ERC20 properties
simon-something marked this conversation as resolved.
Show resolved Hide resolved
(https://github.com/crytic/properties?tab=readme-ov-file#erc20-tests)

# Unit for the math libs (BNum and BMath):

btoi should always return the floor(a / BONE) == (a - a%BONE) / BONE
bfloor should always return (a - a % BONE)
simon-something marked this conversation as resolved.
Show resolved Hide resolved
badd should be commutative
badd should be associative
0 should be identity for badd
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

badd should have 0 as identity?

badd result should always be gte its terms
badd should never sum terms which have a sum gt uint max
badd should have bsub as reverse operation

bsub should not be commutative
bsub should not be associative
bsub should have 0 as identity
bsub result should always be lte its terms
bsub should alway revert if b > a (duplicate with previous tho)

bsubSign result should always be negative if b > a
bsubSign result should always be positive if a > b
bsubSign result should always be 0 if a == b

bmul should be commutative
bmul should be associative
bmul should be distributive
1 should be identity for bmul
0 should be absorbing for mul
bmul result should always be gte a and b

bdiv should be bmul reverse operation // <-- unsolved
1 should be identity for bdiv
bdiv should revert if b is 0 // <-- impl with wrapper to have low lvl call
bdiv result should be lte a

bpowi should return 1 if exp is 0
0 should be absorbing if base
1 should be identity if base
1 should be identity if exp
bpowi should be distributive over mult of the same base x^a * x^b == x^(a+b)
bpowi should be distributive over mult of the same exp a^x * b^x == (a*b)^x
power of a power should mult the exp (x^a)^b == x^(a*b)

bpow should return 1 if exp is 0
0 should be absorbing if base
1 should be identity if base
1 should be identity if exp
bpow should be distributive over mult of the same base x^a * x^b == x^(a+b)
bpow should be distributive over mult of the same exp a^x * b^x == (a*b)^x
power of a power should mult the exp (x^a)^b == x^(a*b)


bpowApprox

calcOutGivenIn

calcOutGivenIn should be inv with calcInGivenOut

calcInGivenOut

calcPoolOutGivenSingleIn

calcPoolOutGivenSingleIn should be inv with calcSingleInGivenPoolOut

calcSingleInGivenPoolOut

calcSingleOutGivenPoolIn

calcSingleOutGivenPoolIn should be inv with calcPoolInGivenSingleOut

calcPoolInGivenSingleOut
5 changes: 5 additions & 0 deletions test/invariant/fuzz/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# https://github.com/crytic/echidna/blob/master/tests/solidity/basic/default.yaml for more options
testMode: assertion
corpusDir: test/invariant/fuzz/corpus/
coverageFormats: ["html","lcov"]
allContracts: true
21 changes: 21 additions & 0 deletions test/invariant/fuzz/external/MockSettler.sol
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
// SPDX-License-Identifier: UNLICENSED
pragma solidity 0.8.25;

import {GPv2Interaction, GPv2Trade, IERC20, ISettlement} from 'interfaces/ISettlement.sol';

contract MockSettler is ISettlement {
function domainSeparator() external view override returns (bytes32) {
return bytes32(hex'1234');
}

function vaultRelayer() external view override returns (address) {
return address(123);
}

function settle(
IERC20[] calldata tokens,
uint256[] calldata clearingPrices,
GPv2Trade.Data[] calldata trades,
GPv2Interaction.Data[][3] calldata interactions
) external {}
}
Loading
Loading