GPT-in-a-Box: Additional info on custom models (nutanix-cloud-native#55)

* add additional info on custom models
yannickstruyf3 · Dec 20, 2023 · e700814 · e700814
1 parent ed83ff7
commit e700814
Show file tree

Hide file tree

Showing 2 changed files with 10 additions and 2 deletions.
diff --git a/docs/gpt-in-a-box/kubernetes/v0.2/custom_model.md b/docs/gpt-in-a-box/kubernetes/v0.2/custom_model.md
@@ -1,7 +1,11 @@
 # Custom Model Support
-We provide the capability to generate a MAR file with custom models and start an inference server using Kubeflow serving.<br />
+In some cases you may want to use a custom model, e.g. a custom fine-tuned model. We provide the capability to generate a MAR file with custom models and start an inference server using Kubeflow serving.<br />
 
 ## Generate Model Archive File for Custom Models
+
+!!! note
+    The model files should be placed in an NFS share accessible by the Nutanix package. This directory will be passed to the --model_path argument. You'll also need to provide the --output path where you want the model archive export to be stored.
+
 To generate the MAR file, run the following:
 ```
 python3 $WORK_DIR/llm/generate.py --skip_download [--repo_version <REPO_COMMIT_ID> --handler <CUSTOM_HANDLER_PATH>] --model_name <MODEL_NAME> --model_path <MODEL_PATH> --output <NFS_LOCAL_MOUNT_LOCATION>

diff --git a/docs/gpt-in-a-box/vm/v0.3/custom_model.md b/docs/gpt-in-a-box/vm/v0.3/custom_model.md
@@ -1,7 +1,11 @@
 # Custom Model Support
-We provide the capability to generate a MAR file with custom model files and start an inference server using it with Torchserve.
+In some cases you may want to use a custom model, e.g. a custom fine-tuned model. We provide the capability to generate a MAR file with custom model files and start an inference server using it with Torchserve.
 
 ## Generate Model Archive File for Custom Models
+
+!!! note
+    The model archive files should be placed in a directory accessible by the Nutanix package, e.g. /home/ubuntu/models/&lt;custom_model_name&gt;/model_files. This directory will be passed to the --model_path argument. You'll also need to provide the --mar_output path where you want the model archive export to be stored.
+
 Run the following command for generating the Model Archive File (MAR) with the Custom Model files :
 ```
 python3 $WORK_DIR/llm/generate.py --skip_download [--repo_version <REPO_VERSION> --handler <CUSTOM_HANDLER_PATH>] --model_name <CUSTOM_MODEL_NAME> --model_path <MODEL_PATH> --mar_output <MAR_EXPORT_PATH>