0 Fork 0

Files

T

History

c4820d05fb [Security] Drop the hard-coded API key from the DS-V4.1 serving config (#1396 )

The Phase 3 YAML shipped a literal api-key. Pass the key at launch instead,
with --api-key or VLLM_API_KEY. The old key is in git history and must be
treated as exposed.

Co-authored-by: yangzhuxinyzx <Susannah_Carterect@gmx.ca>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

2026-11-06 16:55:30 +08:00

attention_benchmarks

…

auto_tune

…

csrc

[Attention] Read device QSA history without protected-cache resolution

2026-10-08 13:53:28 +08:00

cutlass_benchmarks

…

deepseek_v41

[Security] Drop the hard-coded API key from the DS-V4.1 serving config (#1396 )

2026-11-06 16:55:30 +08:00

disagg_benchmarks

…

flashnext_prefill

[CI] Make the Flash-Next merge pass the repository pre-commit

2026-10-22 16:21:39 +08:00

fused_kernels

…

kernels

[Perf] Parallelize Flash-Next expert copies and skip empty KV installs

2026-10-14 04:42:46 +08:00

multi_turn

…

overheads

…

sm70_source_overlay

…

structured_schemas

…

tools

…

__init__.py

…

analyze_dflash2_selector_alignment.py

…

analyze_dsv4_dspark_confidence.py

…

analyze_flashnext_graph_nodes.py

[Perf] Capture and attribute C4 verification graph shapes

2026-10-07 07:11:57 +08:00

analyze_flashnext_kv_cost.py

[Profile] Add an opt-in SM70 round-cost ledger (#1254 )

2026-10-24 00:45:45 +08:00

analyze_flashnext_round_cost.py

[Profile] Add an opt-in SM70 round-cost ledger (#1254 )

2026-10-24 00:45:45 +08:00

analyze_flashnext_round_ledger.py

[Profile] Add an opt-in SM70 round-cost ledger (#1254 )

2026-10-24 00:45:45 +08:00

analyze_sm70_draft_vocab_coverage.py

…

analyze_sm70_dynamic_draft_vocab.py

…

analyze_sm70_mtp4_phases.py

…

analyze_sm70_mtp_draft_nodes.py

…

analyze_sm70_mtp_gate.py

…

analyze_sm70_mtp_nsys.py

…

analyze_sm70_nsys_replay_gaps.py

…

analyze_spec_alignment_dumps.py

…

backend_request_func.py

…

benchmark_batch_invariance.py

…

benchmark_block_pool.py

…

benchmark_dflash2_concurrent_rounds.py

…

benchmark_dspark_api.py

…

benchmark_dspark_stream.py

…

benchmark_dsv4_dspark_long_context.py

…

benchmark_dsv4_gsm8k_api.py

…

benchmark_dsv4_quality_api.py

…

benchmark_flash_v100_9b_awq_e2e.py

…

benchmark_flash_v100_bfla_longbench.py

…

benchmark_flashnext_acceptance.py

[Profile] Add an opt-in SM70 round-cost ledger (#1254 )

2026-10-24 00:45:45 +08:00

benchmark_gguf_dense_projection.py

[Kernel] Reuse dense GGUF weights across M20 verification rows

2026-10-10 02:30:36 +08:00

benchmark_gguf_head_layout.py

…

benchmark_gguf_model.py

…

benchmark_gguf_quantization_inventory.py

…

benchmark_hash.py

…

benchmark_hidden_state_extraction.py

…

benchmark_latency.py

…

benchmark_long_document_qa_throughput.py

…

benchmark_mixed_prefill_latency.py

…

benchmark_ngram_proposer.py

…

benchmark_packed_ple_results.py

[Misc] Pre-commit compliance for the Flash-Next prefill branch

2026-10-13 05:41:13 -07:00

benchmark_ple_metadata_transfers.py

…

benchmark_prefix_block_hash.py

…

benchmark_prefix_caching.py

…

benchmark_prioritization.py

…

benchmark_qwen38_dcp_numerics.py

…

benchmark_qwen38_dcp_quality.py

…

benchmark_serving_structured_output.py

…

benchmark_serving.py

…

benchmark_sm70_79t_cold.py

…

benchmark_sm70_79t_operator.py

…

benchmark_sm70_all_reduce_sum2.py

…

benchmark_sm70_attention_exactness.py

…

benchmark_sm70_awq_compact_metadata.py

…

benchmark_sm70_awq_fp8_decode_pair.py

…

benchmark_sm70_awq_prefill_workspace.py

…

benchmark_sm70_awq_sorted_output.py

…

benchmark_sm70_awq_splitk_reducer_micro.py

…

benchmark_sm70_awq_verifier_micro.py

…

benchmark_sm70_batch_gemm_reuse.py

…

benchmark_sm70_cublas_gemm_algorithms.py

…

benchmark_sm70_cublaslt_algorithms.py

…

benchmark_sm70_custom_all_reduce.py

…

benchmark_sm70_d256_exact_dense.py

…

benchmark_sm70_decode_boundaries.py

…

benchmark_sm70_decode.py

…

benchmark_sm70_deepseek_v4_attention_input_overlap.py

…

benchmark_sm70_deepseek_v4_fp8_dense.py

…

benchmark_sm70_deepseek_v4_fp16_gemv.py

…

benchmark_sm70_deepseek_v4_fused_router.py

…

benchmark_sm70_deepseek_v4_kv_insert.py

…

benchmark_sm70_deepseek_v4_mhc_all_layers.py

…

benchmark_sm70_deepseek_v4_mhc_tile_n.py

…

benchmark_sm70_deepseek_v4_mhc.py

…

benchmark_sm70_dflash2_batched_grouped_verify.py

…

benchmark_sm70_dflash2_fp32_attention.py

…

benchmark_sm70_dflash2_gdn_metadata.py

…

benchmark_sm70_dflash2_gsm8k.py

…

benchmark_sm70_dflash2_sparse_rejection.py

…

benchmark_sm70_draft_lm_head_awq.py

…

benchmark_sm70_dynamic_vocab_gpu_lru.py

…

benchmark_sm70_dynamic_vocab_lm_head.py

…

benchmark_sm70_flashinfer_bm64_pipeline_micro.py

…

benchmark_sm70_flashinfer_fixed_entry.py

…

benchmark_sm70_flashinfer_realshape_ab.py

…

benchmark_sm70_flashinfer_splitkv3_realshape_ab.py

…

benchmark_sm70_flashinfer_volta_mma_micro.py

…

benchmark_sm70_flashqla_indexed_state_cache.py

…

benchmark_sm70_flashqla_prefill_paths.py

…

benchmark_sm70_fp8_fixed_vs_dynamic.py

…

benchmark_sm70_fp8_prefill_exact_dense_blas.py

…

benchmark_sm70_fp8_prefill_tactics.py

…

benchmark_sm70_fp8_qpn8.py

…

benchmark_sm70_gdn_exactness.py

…

benchmark_sm70_gdn_prefill_compare.py

…

benchmark_sm70_glm53_kda_fg_tp8.py

…

benchmark_sm70_glm53_mhc_q8.py

…

benchmark_sm70_glm53_moe_permute_q8.py

…

benchmark_sm70_hmma884_schedule_micro.py

…

benchmark_sm70_hmma_c2a_mincomm.py

…

benchmark_sm70_instruction_latency.py

…

benchmark_sm70_lm_head_top1.py

…

benchmark_sm70_lm_head_top20.py

…

benchmark_sm70_long_quality_suffix.py

…

benchmark_sm70_mixed_decode_rows.py

…

benchmark_sm70_model_tokens.py

…

benchmark_sm70_moe_sum2_runner.py

…

benchmark_sm70_mtp4_round.py

…

benchmark_sm70_mxfp4_moe_active_experts.py

…

benchmark_sm70_native_bm32_allp_crossblock_micro.py

…

benchmark_sm70_native_bm32_allp_p_group_pad_micro.py

…

benchmark_sm70_native_bm32_allp_p_group_skew_micro.py

…

benchmark_sm70_native_bm32_allp_pv_load_fence_micro.py

…

benchmark_sm70_native_bm32_allp_scratch_micro.py

…

benchmark_sm70_native_bm32_full_phase_allp_micro.py

…

benchmark_sm70_native_bm32_full_phase_micro.py

…

benchmark_sm70_native_bm32_pv_panel_micro.py

…

benchmark_sm70_native_bm32_qk_dualwarp_pipeline_micro.py

…

benchmark_sm70_native_bm32_qk_panel_micro.py

…

benchmark_sm70_native_bm32_splitkv_micro.py

…

benchmark_sm70_native_bm64_allp_micro.py

…

benchmark_sm70_nvfp4_gemm_micro.py

…

benchmark_sm70_openai_stream.py

…

benchmark_sm70_prefill_gemm_collective.py

…

benchmark_sm70_qk_b_load_micro.py

…

benchmark_sm70_qsa_host_kv.py

[KV] Reuse protected page slots and skip empty selector tiles

2026-10-07 21:12:17 +08:00

benchmark_sm70_qwen38_baseline.py

…

benchmark_sm70_qwen38_concurrency.py

…

benchmark_sm70_qwen38_distribution.py

…

benchmark_sm70_qwen38_quality.py

…

benchmark_sm70_raw_hmma_probe.py

…

benchmark_sm70_raw_hmma_pv_panel_micro.py

…

benchmark_sm70_raw_hmma_qk_panel_micro.py

…

benchmark_sm70_rejection_sampler_micro.py

…

benchmark_sm70_serving_quality.py

…

benchmark_sm70_sglang_push_chain.py

…

benchmark_sm70_source_inventory.py

…

benchmark_sm70_sparse_q_rejection.py

…

benchmark_sm70_tile_runtime.py

…

benchmark_sm70_tool_datasets.py

…

benchmark_sm70_tool_protocol.py

…

benchmark_sm70_tp2_compact_verifier_logits.py

…

benchmark_sm70_tp2_dynamic_vocab_proposal.py

…

benchmark_sm70_tp2_vocab_row_redistribution.py

…

benchmark_sm70_tp2_w4_static_vocab_proposal.py

…

benchmark_sm70_tp4_fused_norm_collective.py

…

benchmark_sm70_tp4_mtp5_push_allreduce.py

…

benchmark_sm70_tp8_hierarchical_allreduce.py

…

benchmark_sm70_triton_attention_schedule.py

…

benchmark_sm70_turbomind_exactness.py

…

benchmark_sm70_turbomind_parity.py

…

benchmark_sm70_turboquant_quality.py

…

benchmark_sm70_wmma_fragment_probe.py

…

benchmark_startup_cache.py

…

benchmark_throughput.py

…

benchmark_topk_topp.py

…

benchmark_utils.py

…

compare_dsv4_quality_results.py

…

compare_flashnext_acceptance.py

…

compare_sm70_dflash2_natural_audit.py

…

compare_sm70_dflash2_quality_matrix.py

…

compare_sm70_dflash2_state_audit.py

…

compare_sm70_dflash2_tensor_dump.py

…

compare_sm70_quality_trace.py

…

compare_sm70_tensor_dumps.py

…

diagnose_sm70_hcx_schedule.py

[Kernel] Overlap HCX peer loads and stream reductions

2026-10-09 18:18:08 +08:00

diagnose_sm70_qsa_schedule.py

[Benchmark] Repeat QSA C4 comparisons in symmetric orders

2026-10-09 20:07:57 +08:00

flashnext_acceptance_prompts.json

…

flashnext_expert_cache_sim.py

[Misc] Pre-commit compliance for the Flash-Next prefill branch

2026-10-13 05:41:13 -07:00

flashnext_expert_trace.py

[Misc] Pre-commit compliance for the Flash-Next prefill branch

2026-10-13 05:41:13 -07:00

host_link_bandwidth.py

[Misc] Pre-commit compliance for the Flash-Next prefill branch

2026-10-13 05:41:13 -07:00

host_uva_copy.py

[Misc] Pre-commit compliance for the Flash-Next prefill branch

2026-10-13 05:41:13 -07:00

prepare_sm70_qwen38_quality_cases.py

…

probe_sm70_prefix_cache.py

…

profile_sm70_dflash2_context_cost.py

…

profile_sm70_fp8_qpn8_gated_pair_ncu.py

…

profile_sm70_fp8_qpn8_ncu.py

…

qwen38_dcp_datasets.py

…

qwen38_distribution_probe.py

…

qwen38_ple_phase_probe.py

…

rank_sm70_tile_runtime_candidates.py

…

README.md

…

replay_sm70_dflash2_packed_state.py

…

run_sm70_quality_speed_matrix.py

…

run_sm70_release_matrix.py

…

run_structured_output_benchmark.sh

…

sm70_dflash2_context_trace.py

…

sm70_dflash2_prefill_route_audit.py

…

sm70_dflash2_prefill_trace.py

…

sm70_dflash2_scalar_q1_probe.py

…

sm70_dflash2_state_audit.py

…

sm70_dflash2_state_layout.py

…

sm70_dflash2_tail_audit.py

…

sm70_mtp_admission_worker.py

…

sm70_mtp_node_trace.py

…

sm70_mtp_phase_events.py

…

sm70_mtp_reference_worker.py

…

sm70_mtp_teacher_forcing.py

…

sm70_qwen38_baseline_worker.py

…

sm70_qwen38_baseline.py

…

sm70_startup_worker.py

…

sm70_teacher_conditions.py

[Bench] Preserve teacher histories across chained comparisons

2026-10-06 02:40:47 +08:00

sm70_teacher_forcing_metrics.py

…

sonnet.txt

…

summarize_sm70_dflash2_long_curve.py

…

verify_sm70_artifact.py

…

README.md

Benchmarks

This directory used to contain vLLM's benchmark scripts and utilities for performance testing and evaluation.

Serving benchmarks: Scripts for testing online inference performance (latency, throughput)
Throughput benchmarks: Scripts for testing offline batch inference performance
Specialized benchmarks: Tools for testing specific features like structured output, prefix caching, long document QA, request prioritization, and multi-modal inference
Dataset utilities: Framework for loading and sampling from various benchmark datasets (ShareGPT, HuggingFace datasets, synthetic data, etc.)

Usage

For detailed usage instructions, examples, and dataset information, see the Benchmark CLI documentation.

For full CLI reference see:

README.md

Benchmarks

Contents

Usage