enabling multi-node serving on Gaudi ray cluster

[vishnumadhu365]

PR enables multi-node serving on Ray cluster. Tested with meta-llama/Meta-Llama-3.1-405B-Instruct
Originally implemented in SW-195705

Steps to test:

1. Install vllm-fork on master and worker nodes
2. Launch Ray cluster.
3. Run benchmark as below (eg: for 2 nodes, 16 Gaudi cards)

export PT_HPU_ENABLE_LAZY_COLLECTIVES=true
export PT_HPUGRAPH_DISABLE_TENSOR_CACHE=0
export GLOO_SOCKET_IFNAME=eth0

python benchmarks/benchmark_throughput.py --backend vllm --input-len 128 --output-len 128 \
--model  meta-llama/Meta-Llama-3.1-405B-Instruct  --device hpu \
--dtype bfloat16 --tensor-parallel-size 16 --num-prompts 10

4. Serve the model

vllm serve  meta-llama/Meta-Llama-3.1-405B-Instruct  --tensor-parallel-size 16  --pipeline-parallel-size 1

Contributors - @jiunnyeu-habana @kzawora-intel , Jiafan Wang , Kobayashi-san , Nishant Agrawal

[kzawora-intel]

code looks good, but please fix the formatting (format.sh included with vllm)

[michalkuligowski]

Commented code, please remove

[michalkuligowski]

Closing for now due to no update on the subject, lets reopen when needed.