Skip to content

Commit

Permalink
Minor update
Browse files Browse the repository at this point in the history
  • Loading branch information
yinggeh committed Aug 15, 2024
1 parent a99d38b commit de8f25b
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 6 deletions.
8 changes: 3 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,11 +110,9 @@ In this case, please install vLLM first. You can do so by running
container with the following commands:

```
TEMP_TRITON_VLLM_DIR=/tmp/vllm_backend
mkdir -p /opt/tritonserver/backends/vllm $TEMP_TRITON_VLLM_DIR
git clone https://github.com/triton-inference-server/vllm_backend.git $TEMP_TRITON_VLLM_DIR
cp -r $TEMP_TRITON_VLLM_DIR/src/* /opt/tritonserver/backends/vllm
rm -rf $TEMP_TRITON_VLLM_DIR
mkdir -p /opt/tritonserver/backends/vllm
git clone https://github.com/triton-inference-server/vllm_backend.git /tmp/vllm_backend
cp -r /tmp/vllm_backend/src/* /opt/tritonserver/backends/vllm
```

## Using the vLLM Backend
Expand Down
2 changes: 1 addition & 1 deletion src/utils/metrics.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ def _log_counter(self, counter, data: Union[int, float]) -> None:
counter.increment(data)

def log(self, stats: VllmStats) -> None:
"""Logs to triton metrics server every iteration.
"""Report stats to Triton metrics server.
Args:
stats: Created by LLMEngine for use by VllmStatLogger.
Expand Down

0 comments on commit de8f25b

Please sign in to comment.