Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

vLLM 0.6.2 rebase #340

Merged
merged 47 commits into from
Sep 27, 2024
Merged

vLLM 0.6.2 rebase #340

merged 47 commits into from
Sep 27, 2024

Conversation

kzawora-intel
Copy link

you know the drill

janimo and others added 30 commits September 23, 2024 14:43
@janimo
[VLM] Fix paligemma, fuyu and persimmon with transformers 4.45 : use …
f2bd246 
…config.text_config.vocab_size (vllm-project#8707)
@dtrifiro @youkaichao
[CI/Build] use setuptools-scm to set __version__ (vllm-project#4738)
ee5f34b 
Co-authored-by: youkaichao <youkaichao@126.com>
@LucasWilkinson @mgoin
@divakar-amd @tlrmchlsmth
[Kernel] (2/N) Machete - Integrate into CompressedTensorsWNA16 and GP…
86e9c8d 
…TQMarlin (vllm-project#7701)

Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Divakar Verma <137818590+divakar-amd@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
@jeejeelee
[Kernel][LoRA] Add assertion for punica sgmv kernels (vllm-project#7585)
9b0e3ec
@russellb
[Core] Allow IPv6 in VLLM_HOST_IP with zmq (vllm-project#8575)
b05f5c9 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
@jiqing-feng
Fix typical acceptance sampler with correct recovered token ids (vllm…
5f7bb58 
…-project#8562)
@alexm-neuralmagic
Add output streaming support to multi-step + async while ensuring Req…
1a2aef3 
…uestOutput obj reuse (vllm-project#8335)
@hongxiayang
[Hardware][AMD] ROCm6.2 upgrade (vllm-project#8674)
530821d
@sroy745
Fix tests in test_scheduler.py that fail with BlockManager V2 (vllm-p…
88577ac 
…roject#8728)
@youkaichao @LunrEclipse
re-implement beam search on top of vllm core (vllm-project#8726)
0250dd6 
Co-authored-by: Brendan Wong <bjwpokemon@gmail.com>
@simon-mo
Revert "[Core] Rename PromptInputs to PromptType, and inputs to…
3185fb0 
… `prompt`" (vllm-project#8750)
@comaniac
[MISC] Skip dumping inputs when unpicklable (vllm-project#8744)
b8747e8
@petersalas
[Core][Model] Support loading weights by ID within models (vllm-proje…
3f06bae 
…ct#7931)
@alex-jw-brooks @DarkLight1337
[Model] Expose Phi3v num_crops as a mm_processor_kwarg (vllm-project#…
8ff7ced 
…8658)

Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
@hanzhi713
[Bugfix] Fix potentially unsafe custom allreduce synchronization (vll…
cc4325b 
…m-project#8558)
@ElizaWszola @mgoin
[Kernel] Split Marlin MoE kernels into multiple files (vllm-project#8661
a928ded 
)

Co-authored-by: mgoin <michael@neuralmagic.com>
@aandyw @DarkLight1337 @ywang96
[Frontend] Batch inference for llm.chat() API (vllm-project#8648)
2529d09 
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
@LucasWilkinson
[Bugfix] Fix torch dynamo fixes caused by replace_parameters (vllm-…
72fc97a 
…project#8748)
@dtrifiro
[CI/Build] fix setuptools-scm usage (vllm-project#8771)
2467b64
@youkaichao
[misc] soft drop beam search (vllm-project#8763)
1e7d5c0
@jeejeelee
[[Misc]Upgrade bitsandbytes to the latest version 0.44.0 (vllm-projec…
13f9f7a 
…t#8768)
@tjohnson31415
[Core][Bugfix] Support prompt_logprobs returned with speculative deco…
01b6f9e 
…ding (vllm-project#8047)

Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
@apatke
[Core] Adding Priority Scheduling (vllm-project#5958)
6da1ab6
@joerunde
[Bugfix] Use heartbeats instead of health checks (vllm-project#8583)
6e0c9d6
@sroy745
Fix test_schedule_swapped_simple in test_scheduler.py (vllm-project#8780
ee777d9 
)
@sasha0552
[Bugfix][Kernel] Implement acquire/release polyfill for Pascal (vllm-…
b452247 
…project#8776)
@sroy745
Fix tests in test_chunked_prefill_scheduler which fail with BlockMana…
fc3afc2 
…ger V2 (vllm-project#8752)
@zifeitong
[BugFix] Propagate 'trust_remote_code' setting in internvl and minicp…
e3dd069 
…mv (vllm-project#8250)
@Isotr0py
[Hardware][CPU] Enable mrope and support Qwen2-VL on CPU backend (vll…
c239536 
…m-project#8770)
@sohamparikh
[Bugfix] load fc bias from config for eagle (vllm-project#8790)
3e073e6
agt and others added 17 commits September 25, 2024 07:49
@agt
[Frontend] OpenAI server: propagate usage accounting to FastAPI middl…
1ac3de0 
…eware layer (vllm-project#8672)
@darthhexx
[Bugfix] Ray 2.9.x doesn't expose available_resources_per_node (vllm-…
3368c3a 
…project#8767)

Signed-off-by: darthhexx <darthhexx@gmail.com>
@wooyeonlee0
[Misc] Fix minor typo in scheduler (vllm-project#8765)
8fae5ed
@hongxiayang
[CI/Build][Bugfix][Doc][ROCm] CI fix and doc update after ROCm 6.2 up…
1c04644 
…grade (vllm-project#8777)
@bnellnm
[Kernel] Fullgraph and opcheck tests (vllm-project#8479)
300da09
@jeejeelee
[[Misc]] Add extra deps for openai server image (vllm-project#8792)
c6f2485
@DefTruth
[VLM][Bugfix] internvl with num_scheduler_steps > 1 (vllm-project#8614)
0c4d2ad
@DarkLight1337
rename PromptInputs and inputs with backward compatibility (vllm-proj…
28e1299 
…ect#8760)
@Abatom
[Frontend] MQLLMEngine supports profiling. (vllm-project#8761)
64840df
@mgoin
[Misc] Support FP8 MoE for compressed-tensors (vllm-project#8588)
873edda
@simon-mo
Revert "rename PromptInputs and inputs with backward compatibility (v…
4f1ba08 
…llm-project#8760) (vllm-project#8810)
@heheda12345 @simon-mo
@CatherineSue @ywang96
[Model] Add support for the multi-modal Llama 3.2 model (vllm-project…
770ec60 
…#8811)

Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
@ywang96
[Doc] Update doc for Transformers 4.45 (vllm-project#8817)
e2c6e0a
@mgoin
[Misc] Support quantization of MllamaForCausalLM (vllm-project#8822)
7193774
@ywang96
[Misc] Update config loading for Qwen2-VL and remove Granite (vllm-pr…
4bb98f2 
…oject#8837)
@kzawora-intel
Merge remote-tracking branch 'upstream/main' into HEAD
fccaca0
@kzawora-intel
Update cpu-test.yml
5ffcfa3
@kzawora-intel kzawora-intel merged commit f347a84 into habana_main Sep 27, 2024
19 checks passed
@michalkuligowski michalkuligowski deleted the private/kzawora/vllm_0_6_2_rebase branch September 27, 2024 08:18
@kzawora-intel kzawora-intel added the habana Issues or PRs submitted by Habana Labs label Sep 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@madamczykhabana madamczykhabana madamczykhabana approved these changes

Assignees
No one assigned
Labels
habana Issues or PRs submitted by Habana Labs
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.