Add resolve_model_relative_to_config_file config option

[Legion2]

Currently relative paths to local models are resolved relative to the triton server process. However when deploying models to a central model registry one may not know in advance where the model files will end up in the local file system of an triton inference server container. Therefore relative paths should be used to point from the model.json to the directory containing the actual model files. The relative paths are maintained when deploying models and downloading them from a registry. To make this work relative paths in model.json must be resolved relative to the model.json file location.

This PR adds a new property (resolve_model_relative_to_config_file) to the model.json which allows to enable this relative resolution behavior. By default the old behavior is maintained.

I added test for the new feature, but do not understand how to execute them and where they are executed in ci.

[dyastremsky]

Thank you for submitting this PR! Can you sign a Contributor License Agreement (CLA) so that we can review it?

@oandreeva-nv, do you have any thoughts on whether this belongs in the vLLM backend or in Triton server to be backend-agnostic? I'm not exactly sure what the "model" parameter is used for here.

[Legion2]

@dyastremsky I will check how I can sign the CLA, you are referring to this?

[nnshah1]

@Legion2 , @oandreeva-nv - Alternately we could change the behavior such that path to the model is always relative to model dir and not server process.

I think I would rather not add the parameter - because it seems in general more valuable to change the behavior to be relative to model.json - I don't know of the rationale for keeping it relative to the server process.

[Legion2]

Then we need a heuristic to decide whenever it is a relative path which needs to be resolved. Because the hugging face registry model path could also be interpreted as a relative path.

[oandreeva-nv]

@Legion2 , I believe vLLM currently can load weights from a local directory, specified as model in `json. Can we close this PR?

[Legion2]

Relative path work, but when I last checked the relative path was resolved to the working dir of the server process. Which makes the config files not portable, because you need to know where the config is placed relative to the server process. But if you want to store the models with config in S3 and someone else downloads and runs the model, the relative path breaks. This issue is about making the paths relative to the config file, so it does not depend on where you place the config and the model.

[oandreeva-nv]

Got it, do you think there might be any potential security concerns?

[nnshah1]

Then we need a heuristic to decide whenever it is a relative path which needs to be resolved. Because the hugging face registry model path could also be interpreted as a relative path.

I think for this we'd want to clearly distinguish local path from hf path - via a scheme such as file:/// for local or vice versa

[nnshah1]

Got it, do you think there might be any potential security concerns?

I think we would still want to ensure the relative path doesn't escape the model repository root if we can - to avoid any ability for relative path traversal - we can make it relative to config - but again check that the fully resolved path is still within a certain prefix

[Legion2]

This only adds the relative path feature, but absolute paths already work out of the box and I don't think there are any checks to validate where the absolute path is pointing to.

[ozancaglayan]

Hi there. What's the progress on this? In my use case, I find it very inconvenient to maintain a file path in model.json files which are part of the S3 structure. I dont know how message passing works between triton server and backends both it would be very good if this could be passed through CLI args or environment variables.