Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hardhat tests: use loadFixture() #89

Merged
merged 29 commits into from
Apr 12, 2024
Merged

Hardhat tests: use loadFixture() #89

merged 29 commits into from
Apr 12, 2024

Conversation

bpierre
Copy link
Contributor

@bpierre bpierre commented Apr 3, 2024

This PR migrates all the Hardhat tests to take advantage of loadFixture(), resulting in a faster execution.

It also:

On my machine, running the entire test suite went from 11mn and 24s to 39s (~18x faster).

# before
pnpm test  809.97s user 47.66s system 125% cpu 11:24.91 total

# after
pnpm test  148.44s user 10.47s system 125% cpu 2:06.47 total

# after (--parallel)
pnpm test  314.67s user 39.39s system 913% cpu 38.771 total

On the GitHub Action, we seem to be going from ~11mn to ~3mn for the tests, and from ~21mn to ~11mn for the test coverage.

@bpierre bpierre changed the title loadFixture() migration attempt Hardhat tests: loadFixture() migration attempt Apr 3, 2024
This is a test of migrating to loadFixture() using TroveManagerTest only
for now.

The migration consists of converting beforeEach() into a fixture, which
is recommended by Hardhat mostly for performance reasons [1].

On my computer, this test went from ~54s to ~18s (~67% faster).

Before:

pnpm test test/TroveManagerTest.js  53.90s user 6.48s system 145% cpu 41.495 total

After:

pnpm test test/TroveManagerTest.js  17.98s user 1.81s system 138% cpu 14.309 total

[1] https://hardhat.org/hardhat-runner/docs/guides/test-contracts#using-fixtures
…Test

This ocmmit also adds the createDeployAndFundFixture() utility.
@bpierre bpierre changed the title Hardhat tests: loadFixture() migration attempt Hardhat tests: loadFixture() migration Apr 4, 2024
@bpierre bpierre marked this pull request as ready for review April 9, 2024 00:09
describe("Test", async () => {
testCorpus();
});
testCorpus();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the describe("test") but kept the testCorpus() call to not mess up with the indentation in this PR.

@bpierre bpierre changed the title Hardhat tests: loadFixture() migration Hardhat tests: use loadFixture() Apr 9, 2024
It’s probably better to not rely on `accounts[0]` to know if all
accounts have been funded, since `accounts` is dynamic.
Comment on lines -4 to -7
// const areAccountsAlreadyFunded = await token
// .balanceOf(accounts[0])
// .gte(_1e36Str);
// if (areAccountsAlreadyFunded) return;
Copy link
Contributor Author

@bpierre bpierre Apr 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It’s probably better to not rely on accounts[0] to know if all accounts have been funded, since accounts is dynamic.

Comment on lines -10 to +6
return accounts.forEach(async (account) => {
await token.mint(account, _1e36Str);
});
return Promise.all(
accounts.map((account) => (
token.mint(account, String(10n ** 36n))
))
);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably the cause of the remaining finicky tests. I think what was happening was that the tests were getting executed before the accounts funding had finished, which would also explain why it became more frequent once the tests execution started to accelerate.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! If one wants to run async functions in sequence the old fashioned way, it has to be a regular loop:

for (const account of accounts) {
  await token.mint(account, _1e36Str);
}

(Since we don't use the return values anyway).

I also like Promise.all() more. There's a subtle difference, but for us it doesn't matter. If we were on a real chain, the Promise.all() version would try to fire a whole bunch of network requests all at once, which might not be a good idea. Doesn't matter in hardhat though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes true 😅, on a real network we could have some utility to limit the number of promises running concurrently.

@danielattilasimon
Copy link
Collaborator

How come there are fewer tests than on latest main?

Main:
https://github.com/liquity/bold/actions/runs/8644085791/job/23698492242

  416 passing (10m)
  101 pending

This PR:
https://github.com/liquity/bold/actions/runs/8634782943/job/23671029495

  395 passing (2m)
  45 pending

@bpierre
Copy link
Contributor Author

bpierre commented Apr 11, 2024

How come there are fewer tests than on latest main?

Excellent question 😅 After investigating it was caused by some missing imports, and hardhat seems to silence runtime errors for some reason? 🤔

It’s fixed now, thanks for noticing it 🙏 77b757b

# main (7edba5d)

  408 passing (10m)
  105 pending

# test-fixtures

  408 passing (2m)
  49 pending

Some skipped test cases weren't being counted towards the total.
Although they're currently skipped, it's better to fix this or
we might forget about them in the future.
@danielattilasimon danielattilasimon merged commit 7f3e73f into main Apr 12, 2024
6 checks passed
@danielattilasimon danielattilasimon deleted the test-fixtures branch April 12, 2024 06:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants