3 people 9f970d6288 [Bugfix][SM70] Stop compiling prefill kernels for every new prompt length (#1394) ...

* [Bugfix][SM70] Stop compiling prefill kernels for every new prompt length

_qk_norm_rope and _sm70_qwen35_gdn_split_kernel took the token count as a
constexpr, so each prefill chunk length not seen before compiled two Triton
kernels while the request waited: about 0.34 s of TTFT per new length, and
worker RSS that kept growing while serving.

Pass the length-dependent values as runtime arguments instead. Floating-point
behaviour does not change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* [SM70] Leave Qwen3.8-27B about 1 GiB more VRAM headroom

With vision enabled and maximum-size images the 27B engine peaked at 32.1 GiB of
32.48 GiB. KV at 0.86 holds 5.85x of 256K, more than eight requests need at
usual context lengths, so trade about 1 GiB of KV for headroom.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: yangzhuxinyzx <Susannah_Carterect@gmx.ca>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

2026-11-06 09:48:19 +08:00

..

minimax_h3

[Core] Reuse exact DiT checkpoint tensors before layout preparation

2026-10-04 02:08:28 +08:00

transformers

[ROCm][CI] Fix ROCm LoRA Transformers fallback with full CUDA graphs (#41577)

2026-05-23 04:31:32 +00:00

z_image

[Core] Set high precision CUDA reduction defaults

2026-10-02 22:33:53 +08:00

__init__.py

…

adapters.py

[Bugfix][Pooling] Fix silent weight corruption with buffer-reusing iterators (#39650)

2026-04-13 19:37:27 +00:00

afmoe.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

aimv2.py

…

apertus.py

…

arcee.py

[Model] Fix missing maybe_prefix (#42280)

2026-05-11 09:04:06 +00:00

arctic.py

[MoE] Remove inplace fused experts mechanism (#43727)

2026-05-27 20:00:19 -07:00

aria.py

[Model] Fix missing maybe_prefix (#42280)

2026-05-11 09:04:06 +00:00

audioflamingo3.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

AXK1.py

[Model] Use AutoWeightsLoader for AXK1 (#41901)

2026-05-07 05:40:29 -07:00

aya_vision.py

[BugFix] wire make_empty_intermediate_tensors on AyaVision and Voxtral (#43118)

2026-05-22 05:26:41 -07:00

bagel.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

baichuan.py

…

bailing_moe_linear.py

[Kernel] Porting fuse_minimax_qk_norm to manual fusion (#43410)

2026-05-26 13:16:03 -07:00

bailing_moe_v3.py

[Core] Bind FLA schedules and remaining provider lifecycle controls

2026-10-10 08:11:38 +08:00

bailing_moe.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

bamba.py

…

bee.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

bert_with_rope.py

[MoE] Remove inplace fused experts mechanism (#43727)

2026-05-27 20:00:19 -07:00

bert.py

[Perf][4/n] Eliminate various GPU<->CPU syncs (#42347)

2026-05-19 10:35:54 -04:00

blip2.py

[Model] Fix missing maybe_prefix (#42280)

2026-05-11 09:04:06 +00:00

blip.py

…

bloom.py

…

chameleon.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

chatglm.py

[Misc] Cleanup more configs and processors (#37560)

2026-03-19 15:45:23 +00:00

cheers.py

[Model] Add support for Cheers multimodal model (#38788)

2026-04-02 21:01:40 +08:00

clip.py

[Misc][MM] Remove redundant code in CLIPAttention (#43046)

2026-05-19 08:43:16 +00:00

cohere2_moe.py

[Models] Cohere Eagle + fix to Cohere MoE (#42078)

2026-05-08 21:46:26 -07:00

cohere2_vision.py

[BugFix] support PP for Cohere vision model (#42819)

2026-05-18 11:12:06 -07:00

cohere_asr.py

[Model] Fix missing maybe_prefix (#42280)

2026-05-11 09:04:06 +00:00

cohere_eagle.py

[Models] Cohere Eagle + fix to Cohere MoE (#42078)

2026-05-08 21:46:26 -07:00

colbert.py

[Model] Add LFM2-ColBERT-350M support (#37528)

2026-03-20 14:57:57 +00:00

collective_contracts.py

[Core] Bind graph and collective policies to engine resources

2026-10-10 01:02:39 +08:00

colmodernvbert.py

[Model] Update ColModernVBERT to support latest HF checkpoint (#39307)

2026-04-09 10:48:51 +08:00

colpali.py

…

colqwen3_5.py

[Model] Add ColQwen3.5 4.5B support (#36887)

2026-03-17 21:17:02 +00:00

colqwen3.py

…

commandr.py

…

config.py

[Core] Bound Q8 MTP prefill HC residuals with shared row stages

2026-10-15 07:47:03 +08:00

conformer_encoder.py

[model] support FireRedLID (#39290)

2026-04-10 08:43:58 +00:00

cosmos3.py

Add Cosmos3 Reasoner model (#43356)

2026-05-28 09:43:55 -07:00

dbrx.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

deepencoder2.py

[PluggableLayer][MM] Add PluggableLayer for CustomQwen2Decoder (#37293)

2026-03-20 06:24:07 +00:00

deepencoder.py

…

deepseek_eagle3.py

[Bugfix] Apply fc_norm in Eagle3DeepseekV2 combine_hidden_states (#43482)

2026-05-26 00:46:10 -07:00

deepseek_eagle.py

[Model] Fix missing maybe_prefix (#42280)

2026-05-11 09:04:06 +00:00

deepseek_mtp.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

deepseek_ocr2.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

deepseek_ocr.py

[Release] Restore public source snapshot to v1.3.0 (#329)

2026-08-26 15:18:10 +08:00

deepseek_v2.py

[Bugfix] Retain sparse attention policy during layer construction

2026-10-04 01:30:42 +08:00

deepseek_vl2.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

dflash_sm70.py

[CI] Fix GLM DFlash pre-commit gates

2026-09-01 12:58:56 +08:00

dots1.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

dots_ocr.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

eagle2_5_vl.py

[2/3] Refactor InternVL-based processors (#37324)

2026-03-18 22:22:19 +08:00

ernie45_moe.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

ernie45_vl_moe.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

ernie45_vl.py

[Model] Use mm_features for Ernie-4.5 VL M-RoPE (#39753)

2026-04-14 01:11:52 -07:00

ernie45.py

…

ernie_mtp.py

…

ernie.py

…

exaone4_5_mtp.py

Fix EXAONE-4.5 to align with Transformers update (#42246)

2026-05-11 10:25:31 +00:00

exaone4_5.py

Fix EXAONE-4.5 to align with Transformers update (#42246)

2026-05-11 10:25:31 +00:00

exaone4.py

Add EXAONE-4.5 (#39388)

2026-04-09 20:53:26 -07:00

exaone_moe_mtp.py

Add EXAONE-4.5 (#39388)

2026-04-09 20:53:26 -07:00

exaone_moe.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

exaone.py

…

extract_hidden_states.py

[Spec Decode] Support hybrid attention models in extract_hidden_states (#39949)

2026-05-13 10:45:53 -07:00

fairseq2_llama.py

…

falcon_h1.py

[Model] Use AutoWeightsLoader for FalconH1 (#39092)

2026-04-07 16:25:17 +08:00

falcon.py

[Misc] Cleanup more configs and processors (#37560)

2026-03-19 15:45:23 +00:00

fireredasr2.py

[Audio] Bundle get_generation_prompt() params into SpeechToTextParams (#36268)

2026-04-22 12:24:18 +08:00

fireredlid.py

[model] support FireRedLID (#39290)

2026-04-10 08:43:58 +00:00

flex_olmo.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

funasr.py

support hotwords for FunASR model (#39674)

2026-04-22 02:25:06 -07:00

funaudiochat.py

[Core] Add SM70 V100 migration paths

2026-06-14 13:19:58 +08:00

fuyu.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

gemma2.py

…

gemma3_mm.py

[Perf][4/n] Eliminate various GPU<->CPU syncs (#42347)

2026-05-19 10:35:54 -04:00

gemma3.py

[Models][Gemma3/Gemma4] Support hidden_act variants in gated MLP (#40588)

2026-05-08 11:29:11 -07:00

gemma3n_audio_utils.py

…

gemma3n_mm.py

gemma3 multi-gpu bug-fix (#42630)

2026-05-15 02:32:05 -07:00

gemma3n.py

…

gemma4_mm.py

[Release] Restore public source snapshot to v1.3.0 (#329)

2026-08-26 15:18:10 +08:00

gemma4_mtp.py

[Spec Decode] Add Gemma4 MTP speculative decoding support (#41745)

2026-05-06 22:39:29 +08:00

gemma4.py

[Bugfix][Model] Gemma4 MoE routing closure captures per_expert_scale, breaking functional_call substitution (#42250)

2026-05-13 17:43:12 +00:00

gemma.py

…

glm4_1v.py

[feat] add GlmgaProcessor specific logits in glm4_1v.py (#43575)

2026-05-29 02:56:02 +00:00

glm4_moe_lite_mtp.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

glm4_moe_lite.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

glm4_moe_mtp.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

glm4_moe.py

[Bugfix] Fix GLM4-MoE weight loading for NVFP4 quantized checkpoints (#41755)

2026-05-07 05:55:52 -07:00

glm4.py

…

glm4v.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

glm_ocr_mtp.py

…

glm_ocr.py

[bug-fix] GLM OCR Patch Merger context_dim (#37962)

2026-03-26 05:11:21 -07:00

glm.py

…

glmasr_utils.py

…

glmasr.py

[Audio] Bundle get_generation_prompt() params into SpeechToTextParams (#36268)

2026-04-22 12:24:18 +08:00

gpt2.py

[Bugfix][Model] Fix GPT2ForSequenceClassification sub-module prefix (#43579)

2026-05-25 22:43:19 -07:00

gpt_bigcode.py

…

gpt_j.py

…

gpt_neox.py

…

gpt_oss.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

granite4_vision.py

[Model] Fix missing maybe_prefix (#42280)

2026-05-11 09:04:06 +00:00

granite_speech.py

Reduce memory usage for granite_speech. (#42933)

2026-05-25 14:12:57 +08:00

granite.py

…

granitemoe.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

granitemoehybrid.py

…

granitemoeshared.py

…

graph_contract.py

[Perf][SM70][DeepSeek-V4] Keep the cuBLAS decode indexer above 131,072 tokens of context (#1222)

2026-10-28 10:28:54 +08:00

gritlm.py

[Perf] Remove redundant device copies for CPU-only pooling token IDs, 48.9% E2E throughput improvement (#38139)

2026-03-29 18:12:50 +00:00

grok1.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

h2ovl.py

[Misc] Clean up processing logic (#37541)

2026-03-19 13:30:20 +00:00

hunyuan_v1.py

[Model] Fix missing maybe_prefix (#42280)

2026-05-11 09:04:06 +00:00

hunyuan_vision.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

hy_v3_mtp.py

[Model] Support Hy3 preview (#40681)

2026-04-23 22:08:26 +08:00

hy_v3.py

[Model] Support Hy3 preview (#40681)

2026-04-23 22:08:26 +08:00

hyperclovax_vision_v2.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

hyperclovax_vision.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

hyperclovax.py

…

idefics2_vision_model.py

…

idefics3.py

[Perf][4/n] Eliminate various GPU<->CPU syncs (#42347)

2026-05-19 10:35:54 -04:00

interfaces_base.py

[Model] Deprecate the score task (this will not affect users). (#37537)

2026-03-20 08:07:56 +00:00

interfaces.py

[Memory] Reduce SM70 graph scratch and DFlash active peak

2026-09-23 09:58:34 +08:00

intern_vit.py

[Model] Add torch.compile support for InternVL vision encoder (#38049)

2026-03-25 23:52:29 -07:00

internlm2_ve.py

…

internlm2.py

[Model] Use AutoWeightsLoader for InternLM2 (#38278)

2026-05-26 03:39:26 -07:00

interns1_pro.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

interns1_vit.py

…

interns1.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

interns2_preview.py

[Model] Support InternS2 Preview (#42705)

2026-05-14 21:30:26 -07:00

internvl.py

[Perf][4/n] Eliminate various GPU<->CPU syncs (#42347)

2026-05-19 10:35:54 -04:00

iquest_loopcoder.py

…

isaac.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

jais2.py

…

jamba.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

jina_vl.py

…

jina.py

Add Jina Embeddings v5 model support (fixes #38633) (#39575)

2026-04-16 06:37:10 +00:00

kanana_v.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

keye_vl1_5.py

[Model] Use mm_features for Keye-VL and Keye-1.5-VL M-RoPE (#39869)

2026-04-16 02:16:06 -07:00

keye.py

[Model] Fix missing maybe_prefix (#42280)

2026-05-11 09:04:06 +00:00

kimi_audio.py

fix: propagate revision/code_revision pins to all artifact boundaries (#42616)

2026-05-15 02:31:54 -07:00

kimi_k25_vit.py

[Bug] Fix kimi dtype issue with mm_projector_forward (#42081)

2026-05-11 11:45:24 -04:00

kimi_k25.py

[BugFix] Kimi-K2.5: skip vision tower dtype conversion when using quantization (#42869)

2026-05-18 05:07:16 +00:00

kimi_linear.py

[Attention] Mamba attention module refactor (#41126)

2026-05-22 17:13:12 +08:00

kimi_vl.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

laguna.py

[MoE Refactor] Introduce RoutedExperts alias for FusedMoE and don't store SharedExperts in MK (#40735)

2026-05-12 13:37:44 -04:00

lfm2_moe.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

lfm2_siglip2.py

[Model] Fix missing maybe_prefix (#42280)

2026-05-11 09:04:06 +00:00

lfm2_vl.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

lfm2.py

Fix Plamo 2/3 & LFM2 for Transformers v5 (#38090)

2026-03-25 12:29:49 +00:00

lightonocr.py

[Misc] Cleanup more configs and processors (#37560)

2026-03-19 15:45:23 +00:00

llama4_eagle.py

[Model] Fix missing maybe_prefix (#42280)

2026-05-11 09:04:06 +00:00

llama4.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

llama_eagle3.py

fix(eagle3): read norm_before_fc from eagle_config for NVIDIA checkpoint (#42143)

2026-05-23 08:21:34 +00:00

llama_eagle.py

[Model] Fix missing maybe_prefix (#42280)

2026-05-11 09:04:06 +00:00

llama.py

Replace shape_invariants with simpler apprach in dynamic_arg_dims utilizing shape_id property. (#36194)

2026-04-29 18:32:15 +00:00

llava_next_video.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

llava_next.py

[Perf][4/n] Eliminate various GPU<->CPU syncs (#42347)

2026-05-19 10:35:54 -04:00

llava_onevision.py

[Perf][4/n] Eliminate various GPU<->CPU syncs (#42347)

2026-05-19 10:35:54 -04:00

llava.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

longcat_flash_mtp.py

…

longcat_flash.py

Refractor longcat loading to use AutoWeightsLoader (#41448)

2026-05-01 09:07:23 +00:00

mamba2.py

…

mamba.py

…

medusa.py

…

midashenglm.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

mimo_audio.py

[Model] Add MiMo-V2.5 support (#40967)

2026-04-27 13:26:51 +00:00

mimo_mtp.py

…

mimo_v2_mtp.py

[SpecDecoding] extend mtp support for mimo 2.5 (#41905)

2026-05-09 18:22:59 +00:00

mimo_v2_omni.py

[Model] Add MiMo-V2.5 support (#40967)

2026-04-27 13:26:51 +00:00

mimo_v2.py

[Model] update for mimo v25 (#41029)

2026-04-27 21:52:54 -07:00

mimo.py

…

minicpm3.py

…

minicpm_eagle.py

…

minicpm.py

[MoE] Remove inplace fused experts mechanism (#43727)

2026-05-27 20:00:19 -07:00

minicpmo.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

minicpmv4_6.py

[Model] Fix MiniCPM-V 4.6 vit_merger qkv weight loading (#43213)

2026-05-21 22:44:06 -07:00

minicpmv.py

[Bugfix] fix device mismatch in MiniCPM-o-4_5 resampler (#43194)

2026-05-26 13:12:50 +08:00

minimax_m2.py

[Kernel] Porting fuse_minimax_qk_norm to manual fusion (#43410)

2026-05-26 13:16:03 -07:00

minimax_text_01.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

minimax_vl_01.py

…

mistral3.py

[Bugfix][Model] Fix Devstral Small 2 HF format weight loading (#39293)

2026-04-14 10:11:18 +00:00

mistral_eagle.py

[Model] Fix missing maybe_prefix (#42280)

2026-05-11 09:04:06 +00:00

mistral_large_3_eagle.py

…

mistral_large_3.py

…

mistral.py

…

mixtral.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

mllama4.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

mlp_speculator.py

[Refactor] Remove unused dead code (#38842)

2026-04-06 11:52:05 -04:00

modernbert.py

…

module_mapping.py

…

molmo2.py

[Model] Fix missing maybe_prefix (#42280)

2026-05-11 09:04:06 +00:00

molmo.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

moondream3.py

[Model] Add Moondream3 model support(only query and caption skills) (#32325)

2026-05-01 10:06:48 +08:00

moonvit.py

…

mpt.py

…

musicflamingo.py

Update to transformers v5 (#30566)

2026-04-15 16:29:15 -07:00

nano_nemotron_vl.py

Avoid silent weights corruption when loading Nemotron Nano VL with reusable-buffer loaders like runai distributed streaming (#42244)

2026-05-11 12:03:14 +00:00

nemotron_h_mtp.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

nemotron_h.py

Add LM head quantization support for ModelOpt (#42124)

2026-05-26 09:21:05 -07:00

nemotron_nas.py

…

nemotron_parse.py

[Bugfix] Fix LM detection for Nemotron Parse (#42641)

2026-05-14 23:42:10 +08:00

nemotron_vl.py

[Model][Bugfix] Rename weight_mapper to hf_to_vllm_mapper in LlamaNemotronVL pooling models (#43581)

2026-05-28 03:32:22 -07:00

nemotron.py

[Misc] Cleanup more configs and processors (#37560)

2026-03-19 15:45:23 +00:00

nvlm_d.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

olmo2.py

Various Transformers v5 fixes (#38127)

2026-03-26 00:10:08 +00:00

olmo_hybrid.py

[Attention] Mamba attention module refactor (#41126)

2026-05-22 17:13:12 +08:00

olmo.py

…

olmoe.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

opencua.py

…

openpangu_mtp.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

openpangu_vl.py

[Quant] Consolidate GPTQ: rename gptq_marlin.py to auto_gptq.py (#38288)

2026-05-15 08:25:52 +08:00

openpangu.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

openvla.py

[Model] Openvla support (#42654)

2026-05-19 08:17:42 -07:00

opt.py

…

orion.py

…

ouro.py

…

ovis2_5.py

[Model] Fix missing maybe_prefix (#42280)

2026-05-11 09:04:06 +00:00

ovis.py

[Model] Fix missing maybe_prefix (#42280)

2026-05-11 09:04:06 +00:00

paddleocr_vl.py

[Model] Fix missing maybe_prefix (#42280)

2026-05-11 09:04:06 +00:00

paligemma.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

parakeet.py

Bugfix: Parakeet: .conv.pointwise/depthwise_conv1/2.bias weigths can exist even if convolution_bias=False (#40007)

2026-04-16 23:22:05 +00:00

param2moe.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

persimmon.py

…

phi3.py

…

phi3v.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

phi4mm_audio.py

[Perf][4/n] Eliminate various GPU<->CPU syncs (#42347)

2026-05-19 10:35:54 -04:00

phi4mm_utils.py

…

phi4mm.py

[ROCm][CI] Fix ROCm multimodal Qwen2.5-VL activation compile and Phi4MM ragged image mask handling (#43647)

2026-05-26 19:53:34 -07:00

phi4siglip.py

[Model] Add Phi4ForCausalLMV for microsoft/Phi-4-reasoning-vision-15B (#38306)

2026-04-02 21:14:57 -07:00

phi.py

…

phimoe.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

pixtral.py

[Bugfix] Preserve Pixtral vision loading with Transformers 5

2026-10-04 15:20:46 +08:00

plamo2.py

[Misc] Replace mamba_type string literals with MambaAttentionBackendEnum (#41188)

2026-05-11 03:59:36 +00:00

plamo3.py

Fix Plamo 2/3 & LFM2 for Transformers v5 (#38090)

2026-03-25 12:29:49 +00:00

qianfan_ocr.py

[Model] support Qianfan-OCR model (#40136)

2026-05-05 10:51:25 +00:00

qwen2_5_omni_thinker.py

[Perf][4/n] Eliminate various GPU<->CPU syncs (#42347)

2026-05-19 10:35:54 -04:00

qwen2_5_vl.py

Adjust design around encoder_cudagraph_forward (#42288)

2026-05-29 03:02:52 +00:00

qwen2_audio.py

[Perf][4/n] Eliminate various GPU<->CPU syncs (#42347)

2026-05-19 10:35:54 -04:00

qwen2_moe.py

[Perf][SM70] Flash-Next: concurrent MTP4 verification batches (#1369)

2026-11-05 22:45:28 +08:00

qwen2_rm.py

…

qwen2_vl.py

Adjust design around encoder_cudagraph_forward (#42288)

2026-05-29 03:02:52 +00:00

qwen2.py

[Core] Select SM70 AWQ through the linear kernel framework

2026-10-03 01:02:28 +08:00

qwen3_5_mtp.py

[Core] Freeze fallback attention warmup and model provider controls

2026-10-10 07:21:21 +08:00

qwen3_5.py

[Bugfix][SM70] Stop compiling prefill kernels for every new prompt length (#1394)

2026-11-06 09:48:19 +08:00

qwen3_asr_forced_aligner.py

[Feature] Add Qwen3-ForcedAligner support via token classification pooling (#35367)

2026-03-29 00:27:52 +00:00

qwen3_asr_realtime.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

qwen3_asr.py

[Audio] Bundle get_generation_prompt() params into SpeechToTextParams (#36268)

2026-04-22 12:24:18 +08:00

qwen3_dflash2.py

[Core] Resolve runtime policy per engine at ordered defaults

2026-10-09 22:42:41 +08:00

qwen3_dflash.py

[Core] Bind provider policy and native resources to engine lifecycle

2026-10-10 06:53:34 +08:00

qwen3_moe.py

[Core] Add SM70 V100 migration paths

2026-06-14 13:19:58 +08:00

qwen3_next_mtp.py

[Core] Add SM70 V100 migration paths

2026-06-14 13:19:58 +08:00

qwen3_next.py

[Core] Bound original GGUF storage and host KV state on SM70

2026-10-11 01:13:11 +08:00

qwen3_omni_moe_thinker.py

Fix Qwen3-VL and Qwen3-omni-thinker accuracy degradation from deepstack inputs under torch.compile (#43617)

2026-05-27 15:34:08 -07:00

qwen3_vl_moe.py

Fix Weight loading for Qwen3.5-MTP and Qwen3-VL using runai_streamer (#42716)

2026-05-16 17:54:27 -07:00

qwen3_vl.py

[Kernel] Finalize SM70 release paths

2026-07-19 23:44:14 +08:00

qwen3.py

[Feat][Spec Decode] DFlash (#36847)

2026-03-30 15:03:15 -04:00

qwen_vl.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

qwen.py

…

radio.py

Fix Nano Nemotron VL regressions (#38655)

2026-04-03 15:22:06 +08:00

registry.py

[Model] DeepSeek-V4.1-Flash on 4x V100: 512K packed KV, faster prefill, API launch (#1277)

2026-10-27 23:33:35 +08:00

rnj1.py

[Model] Add block-local attention and YaRN for local layers to Gemma3 (#39823)

2026-04-21 23:34:50 -07:00

roberta.py

fix: propagate revision/code_revision pins to all artifact boundaries (#42616)

2026-05-15 02:31:54 -07:00

runtime_defaults.py

[Bugfix][Qwen4Exp] Keep the SM70 decode lane with the vision tower enabled

2026-10-15 04:35:24 -07:00

rvl.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

sarvam.py

[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)

2026-04-23 11:22:34 -04:00

seed_oss.py

…

shared_weights.py

[Core] Isolate prepared layer providers and shared draft weights

2026-10-10 00:05:05 +08:00

siglip2navit.py

[Model] Fix missing maybe_prefix (#42280)

2026-05-11 09:04:06 +00:00

siglip.py

[Model] Fix missing maybe_prefix (#42280)

2026-05-11 09:04:06 +00:00

skyworkr1v.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

smolvlm.py

…

solar.py

…

stablelm.py

…

starcoder2.py

…

step1.py

…

step3_text.py

[Bugfix] Fix Step3 pipeline parallel KeyError for residual tensor (#37622)

2026-05-29 03:04:02 -07:00

step3_vl.py

Adjust design around encoder_cudagraph_forward (#42288)

2026-05-29 03:02:52 +00:00

step3p5_mtp.py

[Bugfix] LoRA: extend expert base_layer loading to Qwen3.5 and Step3.x (#37114)

2026-04-21 14:17:03 +00:00

step3p5.py

[Model]Support Step-3.7-Flash (#43859)

2026-05-28 17:01:48 -07:00

step3p7.py

[Model]Support Step-3.7-Flash (#43859)

2026-05-28 17:01:48 -07:00

step_vl.py

[MM][CG] Enable encoder Cudagraph for Step3VL (#42224)

2026-05-17 20:19:13 -07:00

tarsier.py

[Misc] Cleanup more configs and processors (#37560)

2026-03-19 15:45:23 +00:00

telechat2.py

…

teleflm.py

…

terratorch.py

[Misc] Reorganize inputs (#35182)

2026-03-25 10:22:54 -07:00

ultravox.py

Fix: Propagate pinned model revisions into Ultravox secondary weight loading (#42830)

2026-05-16 17:02:54 +00:00

utils.py

[Bugfix] Honor independent speculative draft load configuration

2026-10-05 00:44:42 +08:00

vision.py

[Perf] FP8 FlashInfer Attn for ViT (#38065)

2026-04-27 13:44:15 +08:00

voxtral_realtime.py

[Audio] Bundle get_generation_prompt() params into SpeechToTextParams (#36268)

2026-04-22 12:24:18 +08:00

voxtral.py

[BugFix] wire make_empty_intermediate_tensors on AyaVision and Voxtral (#43118)

2026-05-22 05:26:41 -07:00

voyage.py

[Model] Use AutoWeightsLoader for Voyage (#42972)

2026-05-22 05:28:23 +00:00

whisper_causal.py

[Attention][V0 Deprecation] Deprecate accept output buffer (#39125)

2026-04-07 17:14:58 -04:00

whisper_utils.py

…

whisper.py

[Audio] Bundle get_generation_prompt() params into SpeechToTextParams (#36268)

2026-04-22 12:24:18 +08:00

zamba2.py

…