admin/1Cat-vLLM
1
0 Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,287 Commits 1 Branch 0 Tags
fc3901beb45817da7cba2247b0aa8fec00b7a659
Commit Graph
114 Commits
Author SHA1 Message Date
yangzhuxinyzx fc3901beb4 [Core] Share SM70 AWQ and FP8 MoE stages and capture engine policy 
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-09 10:54:32 +08:00
yangzhuxinyzxandOpenAI Codex 00c2f2d1c2 [Core] Finish Flash-V100 A3 ownership boundaries 
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-09 09:27:27 +08:00
yangzhuxinyzxandOpenAI Codex 60c41e2b13 [Core] Move constructor environment access behind V100 policy boundary 
Keep all 41 constructor read points and their evaluation order unchanged.

Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-09 00:15:31 +08:00
yangzhuxinyzxandOpenAI Codex e9c3062ba1 [Core] Centralize dynamic Flash-V100 environment reads 
Preserve lookup timing and short-circuit order while reducing direct environment coupling from 119 to 41 and locking the lower ratchet.

Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-09 00:05:00 +08:00
yangzhuxinyzxandGPT-6 58511ea2bc [Arch][Attention] Parameterize grouped admission by KV codec 
Co-Authored-By: GPT-6 <noreply@anthropic.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-08 17:55:20 +08:00
yangzhuxinyzxandGPT-6 71b5fe54aa [Arch][Attention] Register implementation feature hooks 
Co-Authored-By: GPT-6 <noreply@anthropic.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-08 17:24:49 +08:00
yangzhuxinyzxandGPT-6 0a38520587 [Arch][Attention] Register speculative metadata hooks 
Co-Authored-By: GPT-6 <noreply@anthropic.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-08 17:13:12 +08:00
yangzhuxinyzxandGPT-6 fb6b12c5b8 [Arch][Attention] Compose Flash-V100 implementation by responsibility 
Co-Authored-By: GPT-6 <noreply@anthropic.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-08 17:04:09 +08:00
yangzhuxinyzxandGPT-6 c7e4baf3c2 [Arch][Attention] Declare Flash-V100 routes and share XQA admission 
Co-Authored-By: GPT-6 <noreply@anthropic.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-08 16:47:47 +08:00
yangzhuxinyzxandGPT-6 d8aa0bef64 [Arch][Attention] Split Flash-V100 backend by responsibility 
Preserve shared state, legacy imports and patches while moving internal
callers and registry entries to the owning package modules.

Co-Authored-By: GPT-6 <noreply@anthropic.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-08 16:17:10 +08:00
yangzhuxinyzxandClaude Opus 5.5 f11aeb3340 [Arch] Add layering guide and a coupling ratchet for generic modules 
Document the target layering of 1Cat-vLLM on top of upstream vLLM
(platform, formats, operators, dispatch, engine, model adapters) and the
behaviour-preserving refactor roadmap.

Add tools/pre_commit/check_layering.py with a per-file baseline: generic
modules may not gain references to model families, to the SM70 platform
or raw VLLM_* environment reads; reductions are locked in with --update,
and --report prints the remaining debt (182 files: 2325 model, 3964
platform, 335 env references at #1041).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-08 13:19:51 +08:00
yangzhuxinyzx 85a45b3963 [Core] Resolve SM70 DFlash2 verifier policy per engine 
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-03 07:41:12 +08:00
yangzhuxinyzx e415800c77 [Core] Reduce the public SM70 environment control surface 
AI-assisted-by: Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-03 05:55:06 +08:00
yangzhuxinyzx 333217c91a [Core] Move SM70 serialized FP8 dispatch into the linear kernel lifecycle 
AI-assisted-by: Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-03 03:36:20 +08:00
yangzhuxinyzx cdb4157f43 [CI] Keep native environment aliases local to their preceding binding 
AI-assisted-by: Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-03 03:03:02 +08:00
yangzhuxinyzx 5626f5a3f8 Merge remote-tracking branch 'onecat/main' into agent/v100-native-env-metadata-20261002-175711 
AI-assisted-by: Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-03 02:36:47 +08:00
yangzhuxinyzx 2d3679101f [CI/Build] Register native environment controls and check their readers 
AI-assisted-by: Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-03 02:25:29 +08:00
yangzhuxinyzxandCodex 386fa02e89 [Core] Guard AWQ policy compatibility and document aliases 
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-03 01:12:50 +08:00
yangzhuxinyzx 4c31b0940c Merge remote-tracking branch 'onecat/main' into codex/v100-env-metadata-20261002-150828 
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-03 00:27:47 +08:00
yangzhuxinyzxandCodex 4f2371b538 [Core] Document environment policy and register legacy reads 
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-03 00:24:34 +08:00
yangzhuxinyzx 22d4b1a2ee [CI] Format the environment registration scanner 
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-02 23:19:50 +08:00
yangzhuxinyzx d7a23adf3a [CI] Parse environment reads without matching comments or missing aliases 
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-02 23:19:50 +08:00
yangzhuxinyzx e7d21fe232 Merge pull request #711 from Peuqui/check-env-registration 
[CI] Reject new VLLM_* environment reads that bypass envs.py
 2026-10-02 23:13:58 +08:00
PeuquiandClaude 7d12072754 [CI] Reject new VLLM_* environment reads that bypass envs.py 
envs.compile_factors() hashes every variable registered in vllm/envs.py
into the torch.compile cache key. A switch read straight from os.environ
is invisible to it, so flipping such a switch can load an AOT artifact
compiled for the other setting.

Add a pre-commit hook that flags os.getenv / os.environ.get /
os.environ.setdefault / os.environ[...] reads of VLLM_* names under
vllm/ that are not keys of envs.environment_variables. The 137 direct
reads that exist today are listed as a baseline, so the tree stays
green; the list is meant to shrink as switches get registered (a test
fails once a baseline name is registered but still listed).

Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Peuqui <peuqui@github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-02 23:11:38 +08:00
yangzhuxinyzx bc8b673812 [Kernel] Integrate SM70 CT NVFP4 into linear kernel selection 
Resolve QPN2 policy per engine and retain legacy aliases. Preserve native arithmetic and the independent 324-case category route matrix.

AI-assisted-by: Codex (GPT-6)
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-02 21:14:46 +08:00
yangzhuxinyzxandClaude Opus 5.5 835725f6bb Merge remote-tracking branch 'origin/main' into codex/v100-aot-first-reload-20260922-055217 
Resolve the append-only migration-control log by keeping the 2026-09-22
first-reload section followed by later main entries.

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 01:35:07 +08:00
PeuquiandClaude eb3e9b0f86 [CI] Reject new VLLM_* environment reads that bypass envs.py 
envs.compile_factors() hashes every variable registered in vllm/envs.py
into the torch.compile cache key. A switch read straight from os.environ
is invisible to it, so flipping such a switch can load an AOT artifact
compiled for the other setting.

Add a pre-commit hook that flags os.getenv / os.environ.get /
os.environ.setdefault / os.environ[...] reads of VLLM_* names under
vllm/ that are not keys of envs.environment_variables. The 137 direct
reads that exist today are listed as a baseline, so the tree stays
green; the list is meant to shrink as switches get registered (a test
fails once a baseline name is registered but still listed).

Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Peuqui <peuqui@github.com>
 2026-09-27 18:13:53 +02:00
yangzhuxinyzxandOpenAI Codex 6ee93f60ba [Bench] Screen exact SM70 HC batch weight reuse 
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-09-27 02:45:15 +08:00
yangzhuxinyzx ad7e0fb232 [Bugfix] Restore Triton state on the first vLLM AOT reload 
Serialize referenced Triton side-table entries with fixed launch settings and remap them without overwriting other graphs. Ignore frozen pseudo-files in backend hashes so the first reload reuses its original artifacts. Record first-reload timings and the still-open output-parity gate.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
(cherry picked from commit b504a224bc)
 2026-09-22 13:52:50 +08:00
yangzhuxinyzxandCodex 5f02520b56 [Core] Defer first-reload changes until output parity passes 
This reverts commit b504a224bc. Keep the validated channel-FP8 workspace fix in this merge scope; retain the first-reload work and failed experiments in a separate draft follow-up.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-09-22 13:46:54 +08:00
yangzhuxinyzxandCodex b504a224bc [Bugfix] Restore Triton state on the first vLLM AOT reload 
Serialize referenced Triton side-table entries with fixed launch settings and remap them without overwriting other graphs. Ignore frozen pseudo-files in backend hashes so the first reload reuses its original artifacts. Record first-reload timings and the still-open output-parity gate.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-09-22 13:34:28 +08:00
yangzhuxinyzx 58beb3433b Revert "[Release] Restore public source snapshot to v1.3.0 (#329)" 
This reverts commit f4c3451a2b.

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-08-27 22:24:17 +08:00
yangzhuxinyzx f4c3451a2b [Release] Restore public source snapshot to v1.3.0 (#329) 
Freeze the public repository at the exact released v1.3.0 source tree before archiving public development. Ongoing work was mirrored to the personal private repository first; history is preserved without force-pushing.

Target-tag: v1.3.0
Target-tree: 4373fb80cf
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-08-26 15:18:10 +08:00
yangzhuxinyzx 6cdfc4430a [CI/Build] Restore repository pre-commit baseline (#311) 
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-08-26 10:31:35 +08:00
Mohammad Miadh Angkad 158289e0fc [Docs] Fix MLA prefill backend default docs (#43697) 
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
 2026-05-27 10:13:22 +00:00
Bugen ZhaoNick HillEric CurtinDev-X25874Will.houWill.hou
 		39910f2b25 [Rust Frontend] Move code from vllm-frontend-rs (#43283) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Eric Curtin <eric.curtin@docker.com>
Signed-off-by: Dev-X25874 <283057883+Dev-X25874@users.noreply.github.com>
Signed-off-by: Will.hou <1205157517@qq.com>
Signed-off-by: Will.hou <willamhou@ceresman.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Eric Curtin <eric.curtin@docker.com>
Co-authored-by: Dev-X25874 <283057883+Dev-X25874@users.noreply.github.com>
Co-authored-by: Will.hou <1205157517@qq.com>
Co-authored-by: Will.hou <willamhou@ceresman.com>

Please see https://github.com/Inferact/vllm-frontend-rs for full original commit history.
 2026-05-21 17:21:48 -07:00
Lanze LiuandMatthew Bonanni b2c58ee942 [FlashAttn] Fix supports_kv_cache_dtype() accepting unhandled fp8 kv-cache dtype variants (#42685) 
Signed-off-by: Lanze Liu <lanzetech@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-05-15 15:34:59 -04:00
Aaron Hao e0a45f1455 [Feat][RL] IPC weight sync optimizations: multigpu support and chunked packed tensors (#37476) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Signed-off-by: hao-aaron <ahao@anyscale.com>
 2026-05-15 22:53:06 +08:00
CynicDora 256dbcaabf [Feature] Support custom callable proposer backend for speculative decoding (#39487) 
Signed-off-by: 524031910363 <hyzhyzsh@sjtu.edu.cn>
Signed-off-by: CynicDora <hyzhyzsh@sjtu.edu.cn>
 2026-05-13 16:53:01 +00:00
Matthew Bonanni be5983b874 [Docs] Add non-causal support to attention backend docs (#41643) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-05-04 20:35:15 +00:00
Matthew BonanniMichael GoinLucas Wilkinsonmergify[bot]
 		f3fef12350 [Attention] Abstract the MLA prefill backends and eliminate cuDNN (#32623) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-01 13:36:20 -04:00
sychen52 947138b6c2 Add nvfp4 kv cache support (#40177) 
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
 2026-05-01 04:55:16 +00:00
Jiangyun Zhu e8ee2a78db [Attention] use diff kv backend for mimo v2 flash (#40045) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-04-24 11:25:55 +00:00
Martin Hickey 3951d3eacd [MyPy] Enable mypy for vllm/model_executor/layers/ (#40159) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-04-21 20:15:02 -07:00
Vadim Gimpelson 6d85b36a9f Revert #38730 and #38791 (#40032) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
 2026-04-21 11:44:11 -04:00
Harry MellorandRobert Shaw fc645f1acc Add structure to requirements/ directory (#39024) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-04-10 13:46:41 -07:00
Yan Ma ec68d53b2b Add platform manual_seed_all API (#38468) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-04-10 13:43:50 +08:00
Wentao Ye aec18492d0 [CI] Fix mypy for vllm/v1/ops (#39219) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-09 11:06:34 +08:00
Stefano Castagnetta 6183cae1bd [Bugfix] Restrict TRTLLM attention to SM100, fixing GB300 (SM103) hang (#38730) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
 2026-04-01 12:08:40 -07:00
wliao2Harry MellorKunshang Ji
 		4dfad17ed1 replace cuda_device_count_stateless() to current_platform.device_count() (#37841) 
Signed-off-by: Liao, Wei <wei.liao@intel.com>
Signed-off-by: wliao2 <wei.liao@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-31 22:32:54 +08:00