test: Add L0_additional_dependency_dirs #7707
Conversation
fpetrini15
force-pushed
the
fpetrini-additional-dep-dir
branch
from
e01462f to
68dfc7d
Compare
docs/customization_guide/deploy.md
Outdated
|
|
||
| This is an optional Windows feature that enables Triton to search custom | ||
| dependency directories when loading a specific backend. The user can input | ||
| these directories as a string of semicolon-separated paths. These directories |
There was a problem hiding this comment.
Mention that the input must have trailing ";"
| STALE_DEPENDENCY_PATH="C:/ci_test_deps/24.05" | ||
| CUSTOM_DEPENDENCY_DIR=${MODELDIR}/custom_dependency_location | ||
| STALE_DEPENDENCY_DIR=${MODELDIR}/stale_dependency_location | ||
| TRT_MODEL_DIR="C:/tmp/24.07_trt_models" |
There was a problem hiding this comment.
I think this feature is agnostic to the backend, so having TRT models in this test will probably make it more painful to maintain or run locally than necessary. In your follow-up ticket TPRD-244, I would suggest revisiting this.
For example running CI with the BACKENDS="python" or something would have no effect on this test and fail looking for TRT models.
Instead I think TRT backend specific tests could add a test case for this feature if there's a specific TRT related flow it might be used for, and this test could be generic to something easier or more portable like a python model.
This would make it easier to compartmentalize running the tests on scenarios where it would be painful to get the TRT models for, or migrate to other runners, other versions, etc without compatibility issues.
There was a problem hiding this comment.
If the main use case driving the feature is intended for use with TRT backend for now, it's fine - just food for thought.
There was a problem hiding this comment.
I chose TRT for 2 reasons:
- The TRT backend was the motivating backend for this feature (as you say)
- The backend requires implicit dependencies that are expected to exist outside of the backend directory.
The python backend does not depend on any implicit dependencies outside of what exists in its corresponding backend directory and those dependencies must remain in that folder since their path relative to libtriton_python.so matters. I agree that having any other than TRT would be more maintainable, but I'd have to think on how I'd introduce an additional dependency to the python backend specifically for this test.
What does the PR do?
Add testing for new Windows feature that supports additional dependency directories during backend loading.
Checklist
<commit_type>: <Title>Commit Type:
Check the conventional commit type
box here and add the label to the github PR.
Related PRs:
Core: triton-inference-server/core#400
Where should the reviewer start?
Check Core PR first
Test plan:
L0_additional_dependency_dirs--base
[WIP]