feat(anta): Limit concurrency #680
Conversation
carl-baillargeon
force-pushed
the
refactor/runner_limit
branch
from
bf88158 to
d7de654
Compare
|
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
carl-baillargeon
force-pushed
the
refactor/runner_limit
branch
from
b623e90 to
4879120
Compare
|
Conflicts have been resolved. A maintainer will review the pull request shortly. |
|
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
mtache
changed the title
mtache
force-pushed
the
refactor/runner_limit
branch
from
68e259e to
83206e8
Compare
|
Conflicts have been resolved. A maintainer will review the pull request shortly. |
CodSpeed Performance ReportMerging #680 will not alter performanceComparing Summary
|
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
|
Conflicts have been resolved. A maintainer will review the pull request shortly. |
anta/device.py
Outdated
| The limits are set using the following environment variables: | ||
| - ANTA_MAX_CONNECTIONS: Maximum number of allowable connections. | ||
| - ANTA_MAX_KEEPALIVE_CONNECTIONS: Number of allowable keep-alive connections. | ||
| - ANTA_KEEPALIVE_EXPIRY: Time limit on idle keep-alive connections in seconds. |
anta/runner.py
Outdated
| DEFAULT_NOFILE = 16384 | ||
| """Default number of open file descriptors for the ANTA process.""" | ||
| DEFAULT_MAX_CONCURRENCY = 10000 | ||
| """Default maximum number of tests to run concurrently.""" |
There was a problem hiding this comment.
We may need to move all these to constants for simpler way to know where stuff is
anta/runner.py
Outdated
|
|
||
|
|
||
| def adjust_rlimit_nofile() -> tuple[int, int]: | ||
| """Adjust the maximum number of open file descriptors for the ANTA process. | ||
|
|
||
| The limit is set to the lower of the current hard limit and the value of the ANTA_NOFILE environment variable. | ||
|
|
||
| If the `ANTA_NOFILE` environment variable is not set or is invalid, `DEFAULT_NOFILE` is used. | ||
| If the `ANTA_NOFILE` environment variable is not set or is invalid, `DEFAULT_NOFILE` is used (16384). |
There was a problem hiding this comment.
proabbly best to not put the value here - we then forget to update it :p
| The result of each completed test. | ||
| """ | ||
| # NOTE: The `aiter` built-in function is not available in Python 3.9 | ||
| aws = tests_generator.__aiter__() # pylint: disable=unnecessary-dunder-call |
There was a problem hiding this comment.
what does aws stands for here? :)
|
|
||
| # 🚀 Scaling ANTA: A Comprehensive Guide | ||
|
|
||
| **Table of Contents:** |
There was a problem hiding this comment.
do we need this? this is redundant with
| @@ -179,6 +179,7 @@ nav: | |||
| - Debug commands: cli/debug.md | |||
| - Tag Management: cli/tag-management.md | |||
| - Advanced Usages: | |||
| - Scaling ANTA: advanced_usages/scaling.md | |||
There was a problem hiding this comment.
not first - should be last or let's use alphabetical order
|
|
||
| ### Results Management | ||
|
|
||
| ANTA can output results in various formats: |
There was a problem hiding this comment.
do you need to repeat these? I would just point to the NRFU page documentatin and say you use JSON (less maintenance)
| 3. Optional: You can merge the JSON results and generate a JUnit report using the following Python script (requires `junitparser`): | ||
|
|
||
| ```python | ||
| from pathlib import Path |
There was a problem hiding this comment.
please use snippets for this and we will try to test it :)
There was a problem hiding this comment.
Should we provide this by default in ANTA as a feature?
| - 📞 Reach out to your Arista SE for guidance | ||
| - 📝 Document your specific use case on [GitHub](https://github.com/aristanetworks/anta) | ||
| - 🔍 Share your findings with the community |
There was a problem hiding this comment.
repeat from the beginning - probably should be the same (or just a button that goes back to the beginning?:)
|
|
||
| ## 📚 References | ||
|
|
||
| - **Python AsyncIO** |
There was a problem hiding this comment.
need 4 spaces it does not render properly:
tests/benchmark/test_runner.py
Outdated
| @@ -7,8 +7,7 @@ | |||
|
|
|||
| from typing import TYPE_CHECKING | |||
|
|
|||
| from anta.result_manager import ResultManager | |||
| from anta.runner import get_coroutines, prepare_tests | |||
| from anta.runner import prepare_tests | |||
There was a problem hiding this comment.
we can add back get_coroutines for now
|
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
|
Conflicts have been resolved. A maintainer will review the pull request shortly. |
Description
This PR improves the test runner by introducing a generator-based approach for managing test coroutines and setting a configurable limit on the number of concurrent tests.
Instead of loading all test coroutines into a list, the runner now uses a generator to yield tests. This approach prevents memory overload and improves performance when dealing with a large number of tests.
A limit on the number of concurrent tests is introduced to avoid overwhelming the runner. This limit is configurable with an environement variable (hidden).
Implementation:
The generator yields test coroutines, ensuring that only a limited number of tests are scheduled and run concurrently.
Upon reaching the concurrency limit, the runner waits for some tests to complete before scheduling new ones from the generator.
Fixes: #713
Checklist: