admin/1Cat-vLLM
1
0 Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
18,810 Commits 1 Branch 0 Tags
e7d21fe232ba197ffa8a5e0dcbeaad9db465ccd7
Commit Graph
91 Commits
Author SHA1 Message Date
yangzhuxinyzx e7d21fe232 Merge pull request #711 from Peuqui/check-env-registration 
[CI] Reject new VLLM_* environment reads that bypass envs.py
 2026-10-02 23:13:58 +08:00
yangzhuxinyzx bc8b673812 [Kernel] Integrate SM70 CT NVFP4 into linear kernel selection 
Resolve QPN2 policy per engine and retain legacy aliases. Preserve native arithmetic and the independent 324-case category route matrix.

AI-assisted-by: Codex (GPT-6)
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-02 21:14:46 +08:00
yangzhuxinyzxandClaude Opus 5.5 835725f6bb Merge remote-tracking branch 'origin/main' into codex/v100-aot-first-reload-20260922-055217 
Resolve the append-only migration-control log by keeping the 2026-09-22
first-reload section followed by later main entries.

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 01:35:07 +08:00
PeuquiandClaude eb3e9b0f86 [CI] Reject new VLLM_* environment reads that bypass envs.py 
envs.compile_factors() hashes every variable registered in vllm/envs.py
into the torch.compile cache key. A switch read straight from os.environ
is invisible to it, so flipping such a switch can load an AOT artifact
compiled for the other setting.

Add a pre-commit hook that flags os.getenv / os.environ.get /
os.environ.setdefault / os.environ[...] reads of VLLM_* names under
vllm/ that are not keys of envs.environment_variables. The 137 direct
reads that exist today are listed as a baseline, so the tree stays
green; the list is meant to shrink as switches get registered (a test
fails once a baseline name is registered but still listed).

Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Peuqui <peuqui@github.com>
 2026-09-27 18:13:53 +02:00
yangzhuxinyzxandOpenAI Codex 6ee93f60ba [Bench] Screen exact SM70 HC batch weight reuse 
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-09-27 02:45:15 +08:00
yangzhuxinyzx ad7e0fb232 [Bugfix] Restore Triton state on the first vLLM AOT reload 
Serialize referenced Triton side-table entries with fixed launch settings and remap them without overwriting other graphs. Ignore frozen pseudo-files in backend hashes so the first reload reuses its original artifacts. Record first-reload timings and the still-open output-parity gate.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
(cherry picked from commit b504a224bc)
 2026-09-22 13:52:50 +08:00
yangzhuxinyzxandCodex 5f02520b56 [Core] Defer first-reload changes until output parity passes 
This reverts commit b504a224bc. Keep the validated channel-FP8 workspace fix in this merge scope; retain the first-reload work and failed experiments in a separate draft follow-up.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-09-22 13:46:54 +08:00
yangzhuxinyzxandCodex b504a224bc [Bugfix] Restore Triton state on the first vLLM AOT reload 
Serialize referenced Triton side-table entries with fixed launch settings and remap them without overwriting other graphs. Ignore frozen pseudo-files in backend hashes so the first reload reuses its original artifacts. Record first-reload timings and the still-open output-parity gate.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-09-22 13:34:28 +08:00
yangzhuxinyzx 58beb3433b Revert "[Release] Restore public source snapshot to v1.3.0 (#329)" 
This reverts commit f4c3451a2b.

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-08-27 22:24:17 +08:00
yangzhuxinyzx f4c3451a2b [Release] Restore public source snapshot to v1.3.0 (#329) 
Freeze the public repository at the exact released v1.3.0 source tree before archiving public development. Ongoing work was mirrored to the personal private repository first; history is preserved without force-pushing.

Target-tag: v1.3.0
Target-tree: 4373fb80cf
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-08-26 15:18:10 +08:00
yangzhuxinyzx 6cdfc4430a [CI/Build] Restore repository pre-commit baseline (#311) 
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-08-26 10:31:35 +08:00
Mohammad Miadh Angkad 158289e0fc [Docs] Fix MLA prefill backend default docs (#43697) 
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
 2026-05-27 10:13:22 +00:00
Bugen ZhaoNick HillEric CurtinDev-X25874Will.houWill.hou
 		39910f2b25 [Rust Frontend] Move code from vllm-frontend-rs (#43283) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Eric Curtin <eric.curtin@docker.com>
Signed-off-by: Dev-X25874 <283057883+Dev-X25874@users.noreply.github.com>
Signed-off-by: Will.hou <1205157517@qq.com>
Signed-off-by: Will.hou <willamhou@ceresman.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Eric Curtin <eric.curtin@docker.com>
Co-authored-by: Dev-X25874 <283057883+Dev-X25874@users.noreply.github.com>
Co-authored-by: Will.hou <1205157517@qq.com>
Co-authored-by: Will.hou <willamhou@ceresman.com>

Please see https://github.com/Inferact/vllm-frontend-rs for full original commit history.
 2026-05-21 17:21:48 -07:00
Lanze LiuandMatthew Bonanni b2c58ee942 [FlashAttn] Fix supports_kv_cache_dtype() accepting unhandled fp8 kv-cache dtype variants (#42685) 
Signed-off-by: Lanze Liu <lanzetech@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-05-15 15:34:59 -04:00
Aaron Hao e0a45f1455 [Feat][RL] IPC weight sync optimizations: multigpu support and chunked packed tensors (#37476) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Signed-off-by: hao-aaron <ahao@anyscale.com>
 2026-05-15 22:53:06 +08:00
CynicDora 256dbcaabf [Feature] Support custom callable proposer backend for speculative decoding (#39487) 
Signed-off-by: 524031910363 <hyzhyzsh@sjtu.edu.cn>
Signed-off-by: CynicDora <hyzhyzsh@sjtu.edu.cn>
 2026-05-13 16:53:01 +00:00
Matthew Bonanni be5983b874 [Docs] Add non-causal support to attention backend docs (#41643) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-05-04 20:35:15 +00:00
Matthew BonanniMichael GoinLucas Wilkinsonmergify[bot]
 		f3fef12350 [Attention] Abstract the MLA prefill backends and eliminate cuDNN (#32623) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-01 13:36:20 -04:00
sychen52 947138b6c2 Add nvfp4 kv cache support (#40177) 
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
 2026-05-01 04:55:16 +00:00
Jiangyun Zhu e8ee2a78db [Attention] use diff kv backend for mimo v2 flash (#40045) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-04-24 11:25:55 +00:00
Martin Hickey 3951d3eacd [MyPy] Enable mypy for vllm/model_executor/layers/ (#40159) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-04-21 20:15:02 -07:00
Vadim Gimpelson 6d85b36a9f Revert #38730 and #38791 (#40032) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
 2026-04-21 11:44:11 -04:00
Harry MellorandRobert Shaw fc645f1acc Add structure to requirements/ directory (#39024) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-04-10 13:46:41 -07:00
Yan Ma ec68d53b2b Add platform manual_seed_all API (#38468) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-04-10 13:43:50 +08:00
Wentao Ye aec18492d0 [CI] Fix mypy for vllm/v1/ops (#39219) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-09 11:06:34 +08:00
Stefano Castagnetta 6183cae1bd [Bugfix] Restrict TRTLLM attention to SM100, fixing GB300 (SM103) hang (#38730) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
 2026-04-01 12:08:40 -07:00
wliao2Harry MellorKunshang Ji
 		4dfad17ed1 replace cuda_device_count_stateless() to current_platform.device_count() (#37841) 
Signed-off-by: Liao, Wei <wei.liao@intel.com>
Signed-off-by: wliao2 <wei.liao@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-31 22:32:54 +08:00
Lucas Kabela e31915063d [Bugfix] Fix for builtins (forward fix of pytorch/177558) (#37234) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-03-31 01:08:11 +00:00
Andreas Karatzas 43cc5138e5 [ROCm][CI] Fix cross-attention dispatch for encoder-decoder models (#38450) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-28 22:08:03 -07:00
Harry Mellor b3601da6e7 [Mypy] Fix mypy for vllm/model_executor (except vllm/model_executor/layers) (#37904) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-24 17:14:01 +00:00
Wentao Ye 45bd5c8e75 [Mypy] Fix mypy for vllm/config (#37808) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-23 14:33:59 +00:00
Wei Zhao b36adfa349 [Perf] Set Flashinfer sparse MLA as default backend for FP8 kv cache (#37252) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-03-17 20:09:20 +00:00
Isotr0py a836524d20 [Chore] Replace all base64 usages with faster pybase64 package (#37290) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-17 14:44:19 +00:00
Kunshang Ji 747b068136 [Hardware] Replace memory related torch.cuda APIs (#37031) 
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
 2026-03-16 10:24:48 +00:00
Dimitrios BariamisandDimitrios Bariamis cc16b24b17 Update Flashinfer to 0.6.6 (#36768) 
Signed-off-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
Co-authored-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
 2026-03-12 13:19:19 -04:00
Kunshang Ji 53ec16a705 [Hardware] Replace torch.cuda.device_count/current_device/set_device API (#36145) 
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-12 07:57:47 -07:00
Martin HickeyandHarry Mellor 7f1f36bf91 [CI] Fix mypy for vllm/reasoning (#35742) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-12 12:21:33 +00:00
Yan Ma 894843eb25 replace with torch.cuda.device with with torch.accelerator.device_index (#36144) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-03-11 23:12:57 -07:00
Harry Mellor a0f44bb616 Allow markdownlint to run locally (#36398) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-08 20:05:24 -07:00
Wei Zhao 379689d533 [Perf] Support FP8 KV cache for Flashinfer MLA Sparse (#35891) 2026-03-07 13:51:54 -08:00
Kunshang Ji 66a2209645 [Hardware] Replace torch.cuda.synchronize() api with torch.accelerator.synchronize (#36085) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-05 10:36:39 +00:00
Taneem IbrahimandHarry Mellor 1aaec59d79 [MISC] fixed tool_parser mypy errors (#35640) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-04 12:23:12 +00:00
Kunshang JiandHarry Mellor 16d2ad1d38 [Hardware] Replace torch.cuda.empty_cache with torch.accelerator.empty_cache (#30681) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-04 09:49:47 +00:00
Martin HickeyandHarry Mellor 7560d674c9 [CI] Fix mypy for vllm/device allocator (#35518) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-02 15:53:18 +00:00
Lucas WilkinsonMatthew BonanniTyler Michael Smith
 		8b5014d3dd [Attention] FA4 integration (#32974) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2026-03-01 23:44:57 +00:00
Taneem Ibrahim 59d7af9c6c [MISC] Fixing a null reference by removing parallel_utils from mypy EXCLUDE (#35630) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-03-01 09:26:44 -05:00
Aaron Hao 2ce6f3cf67 [Feat][RL][2/2] Native Weight Syncing API: IPC (#34171) 
Signed-off-by: hao-aaron <ahao@anyscale.com>
Signed-off-by: Aaron Hao <ahao@anyscale.com>
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
 2026-02-27 13:45:21 -07:00
Taneem Ibrahim d38cd3dde5 [Misc] Fix mypy errors in vllm/profiler and remove from exclude list (#34959) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-02-20 19:56:33 -08:00
junuxyz c61a98f529 [CI][BugFix] ShellCheck cleanup to remove baseline and preserve runtime behavior (#34514) 
Signed-off-by: junuxyz <216036880+junuxyz@users.noreply.github.com>
 2026-02-17 12:22:56 +00:00
Aneesh Puttur 0b5f9b7204 [CI] Enable mypy import following for vllm/v1/kv_offload (#34639) 
Signed-off-by: Aneesh Puttur <aneeshputtur@gmail.com>
 2026-02-17 09:58:15 +08:00