Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setting enough cache_size_limit for torch.compile warmup #238

Merged
merged 10 commits into from
Sep 25, 2024

Conversation

zehao-intel
Copy link

@zehao-intel zehao-intel commented Sep 4, 2024

Fix the issue that warmup sometimes doesn't work because the default cache_size_limit is only 8 .

Signed-off-by: zehao-intel <zehao.huang@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
vllm/worker/habana_model_runner.py Outdated Show resolved Hide resolved
vllm/worker/habana_model_runner.py Outdated Show resolved Hide resolved
zehao-intel and others added 3 commits September 9, 2024 10:13
Co-authored-by: Andrzej Kotłowski <Andrzej.Kotlowski@intel.com>
Co-authored-by: Andrzej Kotłowski <Andrzej.Kotlowski@intel.com>
vllm/worker/habana_model_runner.py Outdated Show resolved Hide resolved
vllm/worker/habana_model_runner.py Outdated Show resolved Hide resolved
Signed-off-by: zehao-intel <zehao.huang@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
@kzawora-intel
Copy link

please fix formatting (format.sh script) so that static checks pass, otherwise lgtm

Signed-off-by: zehao-intel <zehao.huang@intel.com>
@zehao-intel
Copy link
Author

please fix formatting (format.sh script) so that static checks pass, otherwise lgtm

Thanks for your comments, the format has been fixed now.

Signed-off-by: zehao-intel <zehao.huang@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
@michalkuligowski michalkuligowski merged commit cef2f54 into HabanaAI:habana_main Sep 25, 2024
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants