Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SYCL] fix multi-gpu issue on sycl #8554

Merged
merged 11 commits into from
Jul 25, 2024
Merged

[SYCL] fix multi-gpu issue on sycl #8554

merged 11 commits into from
Jul 25, 2024

Conversation

ClarkChin08
Copy link
Contributor

@ClarkChin08 ClarkChin08 commented Jul 18, 2024 •
edited by airMeng
Loading

fix multi-gpu crash issue by filter the platforms of sycl.

fix multi-gpu issue on sycl
cd296fe 
Signed-off-by: Chen Xi <xi2chen@intel.com>
@github-actions github-actions bot added the SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language label Jul 18, 2024
@ClarkChin08 ClarkChin08 changed the title fix multi-gpu issue on sycl [SYCL] fix multi-gpu issue on sycl Jul 18, 2024
Chen Xi added 3 commits July 18, 2024 07:13
fix some typo
6b4f7b2 
Signed-off-by: Chen Xi <xi2chen@intel.com>
remove unnecessary whitespace
d096de2 
Signed-off-by: Chen Xi <xi2chen@intel.com>
file format issue
bd71cda 
Signed-off-by: Chen Xi <xi2chen@intel.com>
@ClarkChin08 ClarkChin08 mentioned this pull request Jul 18, 2024
Closed
@airMeng
Copy link
Collaborator

airMeng commented Jul 18, 2024

@luoyu-intel

@airMeng
Copy link
Collaborator

airMeng commented Jul 18, 2024

@ClarkChin08 can you attach the measurements results? like llama3-70B on 8 GPUs, memory consumption on each GPU, performance?

@ClarkChin08
Copy link
Contributor Author

This is the llama2-70b memory consumption data and the performance data:
0. run command: ./build/bin/llama-cli -m ../llama-2-70b-chat.Q4_0.gguf -p "how to build a website?" -n 400 -e -ngl 81 -sm layer

  1. memory consumption
    image

image
2. performance
image

sycl::queue can directly use as shared_ptr
6160a76 
Signed-off-by: Chen Xi <xi2chen@intel.com>
OuadiElfarouki
OuadiElfarouki reviewed Jul 18, 2024
View reviewed changes
ggml/src/ggml-sycl/dpct/helper.hpp Outdated Show resolved Hide resolved
ggml/src/ggml-sycl/dpct/helper.hpp Outdated Show resolved Hide resolved
@mofosyne mofosyne added Review Complexity : Medium Generally require more time to grok but manageable by beginner to medium expertise level Review Complexity : High Generally require indepth knowledge of LLMs or GPUs and removed Review Complexity : Medium Generally require more time to grok but manageable by beginner to medium expertise level labels Jul 19, 2024
@ClarkChin08
fix the perf issue of multi-device
e4b86a1 
Signed-off-by: Chen Xi <xi2.chen@intel.com>
@ClarkChin08
fix intel mkl
22c72c5 
Signed-off-by: Chen Xi <xi2.chen@intel.com>
@ClarkChin08
Copy link
Contributor Author

image
This is the new performance table with input=6 and output=32

OuadiElfarouki
OuadiElfarouki approved these changes Jul 24, 2024
View reviewed changes
Copy link
Collaborator

@OuadiElfarouki OuadiElfarouki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good thank you!

@ClarkChin08
add doc for sycl multi-card
fcce873 
Signed-off-by: Chen Xi <xi2.chen@intel.com>
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Jul 25, 2024
@ClarkChin08
add linux part change on doc
8fe8086 
Signed-off-by: Chen Xi <xi2.chen@intel.com>
@ClarkChin08
fix typo
fc76684 
Signed-off-by: Chen Xi <xi2.chen@intel.com>
@airMeng airMeng merged commit ed67bcb into ggerganov:master Jul 25, 2024
53 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NeoZhangJianyu NeoZhangJianyu NeoZhangJianyu left review comments

@luoyu-intel luoyu-intel luoyu-intel approved these changes

@OuadiElfarouki OuadiElfarouki OuadiElfarouki approved these changes

@airMeng airMeng airMeng approved these changes

Assignees
No one assigned
Labels
documentation Improvements or additions to documentation Review Complexity : High Generally require indepth knowledge of LLMs or GPUs SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@ClarkChin08 @airMeng @NeoZhangJianyu @OuadiElfarouki @luoyu-intel @mofosyne