admin/1Cat-vLLM
1
0 Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,769 Commits 1 Branch 0 Tags
main
Commit Graph
145 Commits
Author SHA1 Message Date
claude b4df7b0ea4 [CI] Make the Flash-Next merge pass the repository pre-commit 
Run with --all-files --hook-stage manual, as in CI.

- typos: keep the DSO hash as one token instead of a split identifier.
- clang-format and markdownlint fixes for the prefill notes and the dmv13
  packer.
- mypy: declare the fake processor's tokenizer; annotate a test list.
- Imports: pybase64 in the route capture script; allow pickle in the two
  tests whose subject is a pickling round trip.
- torch.accelerator.synchronize instead of torch.cuda.synchronize in the
  prefill preparation teardown and the expert-cache and MTP Q8 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-22 16:21:39 +08:00
claude d67a10c4ff [Perf] Lower concurrent MTP verification cost on SM70 
Concurrent verification (2-8 requests x 5 rows) now keeps the single-request
fast paths or a grouped equivalent; every change is bitwise equal to the
per-request result.

- QSA shared-key indexer scores several requests in one launch.
- Router, shared expert and PLE spec conv run 5..40 rows in one launch.
- Exact gated RMSNorm covers 64 tokens (was a 9-kernel torch chain at c4).
- Routed gate/up decodes each expert once per few of its routes (IQ3_S,
  IQ4_NL, IQ4_XS) from 10 rows.
- QSA fused side projection chooses its path per call: the decode compile
  range spans 1..40 rows and had frozen the unfused branch.
- Expert-cache lookups publish hits without system-scope fences.
- GDN replay helpers move to a neutral module; the commit-only verifier call
  lives with the native verifier.

54304 GPUs 4-7, MAX_NUM_SEQS=8, verify ms/round:
c1 4K 18.96, 32K 19.56; c2 4K 26.2, 32K 30.9; c4 4K 36.4, 32K 45.2;
c8 4K 62.9.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-21 23:34:52 +08:00
macandClaude Opus 5.5 d836d87d2d [Misc] Pre-commit compliance for the Flash-Next prefill branch 
Register the new VLLM_* switches in vllm/envs.py and read them through
envs; torch.accelerator instead of torch.cuda; regex instead of re; read
the hybrid PLE flag through ple_policy; move the IQ4_NL packed PLE result
helpers to kernels/ple/gguf_packed_result.py; type annotations for lazily
started expert-cache state; ruff/clang-format/markdown fixes. The layering
baseline accepts the opt-in SM70 sparse target sampling hooks in the EAGLE
speculator and its greedy-draft rejection kernel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 05:41:13 -07:00
yangzhuxinyzx 700c891f94 [Core] Reduce host-KV startup and exact MTP storage peaks 
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-11 07:01:42 +08:00
yangzhuxinyzxandCodex e617e9543f [Core] Share GDN operand preparation and prefill lifecycle 
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-10 17:50:13 +08:00
yangzhuxinyzx 276d561f25 [Core] Enforce Phase D ownership and close effective cache aliases 
Assisted-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-10 10:42:39 +08:00
yangzhuxinyzx ec1484d7ec [Core] Close runtime parameter ownership and bind FlashQLA policy 
Assisted-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-10 10:17:08 +08:00
yangzhuxinyzx c4d8716448 [Core] Bind remaining event observers and coalesce proven native scopes 
Assisted-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-10 08:46:29 +08:00
yangzhuxinyzx c43f5d78cb [Core] Bind Marlin overrides through the native engine policy 
Assisted-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-10 08:37:11 +08:00
yangzhuxinyzx 234c56abb2 [Core] Bind FLA schedules and remaining provider lifecycle controls 
Assisted-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-10 08:11:38 +08:00
yangzhuxinyzx e9b9e042a5 [Core] Freeze quantized provider inputs before worker initialization 
Assisted-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-10 07:42:53 +08:00
yangzhuxinyzx 7ec7a210c6 [Core] Freeze fallback attention warmup and model provider controls 
Assisted-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-10 07:21:21 +08:00
yangzhuxinyzx 495fb442a3 [Core] Bind provider policy and native resources to engine lifecycle 
Assisted-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-10 06:53:34 +08:00
yangzhuxinyzx 356b89b4b3 [Core] Bind remaining QSA projection and MoE provider policies 
Assisted-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-10 06:08:04 +08:00
yangzhuxinyzx e512249d05 [Core] Bind 79T prefill policy and native resources per engine 
Assisted-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-10 05:18:51 +08:00
yangzhuxinyzx c8efae47b1 [Core] Bind Flash-V100 policies and workspaces per engine 
Assisted-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-10 04:46:34 +08:00
yangzhuxinyzx 1ca9ea9f87 [Core] Bind diagnostic policies and resources per engine 
Resolve tensor and sampling observations during configuration, share filtering
and dump/capture ownership, and preserve legacy typed MoE adapters. Keep the
DSpark confidence output contract in its graph hash.

Assisted-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-10 03:28:17 +08:00
yangzhuxinyzx e9434be7ac [Core] Capture GDN projection and speculative sampling policies per engine 
Assisted-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-10 02:33:28 +08:00
yangzhuxinyzx bdfce8b79a [Core] Inventory runtime policy and retain evidenced deprecations 
Assisted-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-10 01:54:55 +08:00
yangzhuxinyzxandOpenAI Codex ff4da99893 [Core] Bind graph and collective policies to engine resources 
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-10 01:02:39 +08:00
yangzhuxinyzxandOpenAI Codex 19a2d4e99e [Core] Isolate prepared layer providers and shared draft weights 
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-10 00:05:05 +08:00
yangzhuxinyzxandOpenAI Codex 108da48ef1 [Core] Resolve runtime policy per engine at ordered defaults 
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-09 22:42:41 +08:00
yangzhuxinyzxandOpenAI Codex e045fc8723 [Core] Isolate GDN state owners and unify metadata preparation 
Share request-token preparation and metadata restoration, bind state policy per engine, and delegate model-state fields to backend contracts.

Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-09 21:31:10 +08:00
yangzhuxinyzxandOpenAI Codex 72d8b4d711 [Core] Bind GDN policy and share computation stages 
Preserve existing provider selection, arithmetic and state layouts while binding per-engine tuning and explicit stage contracts.

Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-09 20:57:20 +08:00
yangzhuxinyzxandOpenAI Codex 761e474dd4 [Core] Separate input transfer ownership and target sampling hooks 
Co-authored-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-09 19:14:14 +08:00
yangzhuxinyzx 4ad36d811c [Core] Integrate shared profiling and warmup lifecycle 
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-09 18:44:27 +08:00
yangzhuxinyzxandOpenAI Codex 2f459e30d8 [Core] Share runner profiling and ordered warmup 
Co-authored-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-09 18:30:38 +08:00
yangzhuxinyzx bb1ba51ded [Core] Integrate main policies and isolate HC feature callbacks 
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-09 18:23:23 +08:00
yangzhuxinyzx 3f6395218b [Core] Cache native policy transport outside token execution 
Assisted-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-09 14:32:24 +08:00
yangzhuxinyzx c9d825d5cf [Core] Consolidate prepared SM70 linear and native policy ownership 
Bind linear implementations during preparation, partition native tuning and engine scratch ownership, and retain legacy imports and operator defaults. Extend shared configuration and execution explanations.

Assisted-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-09 13:31:46 +08:00
yangzhuxinyzx 8c08939253 [Core] Share FP4 MoE stages and capture format policy 
Assisted-by: OpenAI Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-09 11:50:23 +08:00
yangzhuxinyzx fc3901beb4 [Core] Share SM70 AWQ and FP8 MoE stages and capture engine policy 
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-09 10:54:32 +08:00
yangzhuxinyzxandOpenAI Codex 00c2f2d1c2 [Core] Finish Flash-V100 A3 ownership boundaries 
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-09 09:27:27 +08:00
yangzhuxinyzxandOpenAI Codex 60c41e2b13 [Core] Move constructor environment access behind V100 policy boundary 
Keep all 41 constructor read points and their evaluation order unchanged.

Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-09 00:15:31 +08:00
yangzhuxinyzxandOpenAI Codex e9c3062ba1 [Core] Centralize dynamic Flash-V100 environment reads 
Preserve lookup timing and short-circuit order while reducing direct environment coupling from 119 to 41 and locking the lower ratchet.

Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-09 00:05:00 +08:00
yangzhuxinyzxandGPT-6 58511ea2bc [Arch][Attention] Parameterize grouped admission by KV codec 
Co-Authored-By: GPT-6 <noreply@anthropic.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-08 17:55:20 +08:00
yangzhuxinyzxandGPT-6 71b5fe54aa [Arch][Attention] Register implementation feature hooks 
Co-Authored-By: GPT-6 <noreply@anthropic.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-08 17:24:49 +08:00
yangzhuxinyzxandGPT-6 0a38520587 [Arch][Attention] Register speculative metadata hooks 
Co-Authored-By: GPT-6 <noreply@anthropic.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-08 17:13:12 +08:00
yangzhuxinyzxandGPT-6 fb6b12c5b8 [Arch][Attention] Compose Flash-V100 implementation by responsibility 
Co-Authored-By: GPT-6 <noreply@anthropic.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-08 17:04:09 +08:00
yangzhuxinyzxandGPT-6 c7e4baf3c2 [Arch][Attention] Declare Flash-V100 routes and share XQA admission 
Co-Authored-By: GPT-6 <noreply@anthropic.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-08 16:47:47 +08:00
yangzhuxinyzxandGPT-6 d8aa0bef64 [Arch][Attention] Split Flash-V100 backend by responsibility 
Preserve shared state, legacy imports and patches while moving internal
callers and registry entries to the owning package modules.

Co-Authored-By: GPT-6 <noreply@anthropic.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-08 16:17:10 +08:00
yangzhuxinyzxandClaude Opus 5.5 f11aeb3340 [Arch] Add layering guide and a coupling ratchet for generic modules 
Document the target layering of 1Cat-vLLM on top of upstream vLLM
(platform, formats, operators, dispatch, engine, model adapters) and the
behaviour-preserving refactor roadmap.

Add tools/pre_commit/check_layering.py with a per-file baseline: generic
modules may not gain references to model families, to the SM70 platform
or raw VLLM_* environment reads; reductions are locked in with --update,
and --report prints the remaining debt (182 files: 2325 model, 3964
platform, 335 env references at #1041).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-08 13:19:51 +08:00
yangzhuxinyzx 85a45b3963 [Core] Resolve SM70 DFlash2 verifier policy per engine 
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-03 07:41:12 +08:00
yangzhuxinyzx e415800c77 [Core] Reduce the public SM70 environment control surface 
AI-assisted-by: Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-03 05:55:06 +08:00
yangzhuxinyzx 333217c91a [Core] Move SM70 serialized FP8 dispatch into the linear kernel lifecycle 
AI-assisted-by: Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-03 03:36:20 +08:00
yangzhuxinyzx cdb4157f43 [CI] Keep native environment aliases local to their preceding binding 
AI-assisted-by: Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-03 03:03:02 +08:00
yangzhuxinyzx 5626f5a3f8 Merge remote-tracking branch 'onecat/main' into agent/v100-native-env-metadata-20261002-175711 
AI-assisted-by: Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-03 02:36:47 +08:00
yangzhuxinyzx 2d3679101f [CI/Build] Register native environment controls and check their readers 
AI-assisted-by: Codex
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-03 02:25:29 +08:00
yangzhuxinyzxandCodex 386fa02e89 [Core] Guard AWQ policy compatibility and document aliases 
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-03 01:12:50 +08:00
yangzhuxinyzx 4c31b0940c Merge remote-tracking branch 'onecat/main' into codex/v100-env-metadata-20261002-150828 
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
 2026-10-03 00:27:47 +08:00