1Cat-vLLM/docs/configuration/env_var_reference.md

Environment variable reference

Defaults describe an unset environment. The declaration is the public getter contract. Effective defaults include per-engine policy and the defaults of legacy direct consumers. Automatic conditions name the policy or consumer that makes the decision. An enabled option still needs its kernel's dtype, layout and native-operator checks. Runtime routing depends on those checks.

Categories distinguish configuration, tuning, experimental, debug and deprecated controls. Prefer engine configuration and VLLM_DISABLED_KERNELS for migrated acceleration paths.

This is the user reference. Internal tuning, diagnostics and compatibility aliases remain in the metadata inventory; they are not daily configuration options. Developers can render that inventory with --include-internal --output /tmp/environment-internals.md. See SM70 environment surface for compatibility and removal plans.

Variable	Category	Description	Declared default	Effective default	Automatic conditions / legacy consumer defaults	Acceleration paths
CMAKE_BUILD_TYPE	configuration	CMake build type If not set, defaults to "Debug" or "RelWithDebInfo" Available options: "Debug", "Release", "RelWithDebInfo"	None	None	None	No SM70 path association
CUDA_HOME	configuration	path to cudatoolkit home directory, under which should be bin, include, and lib directories.	None	None	None	No SM70 path association
CUDA_VISIBLE_DEVICES	configuration	used to control the visible devices in the distributed setting	None	None	None	No SM70 path association
K_SCALE_CONSTANT	configuration	Divisor for dynamic key scale factor calculation for FP8 KV Cache	200	200	None	No SM70 path association
LD_LIBRARY_PATH	configuration	when VLLM_NCCL_SO_PATH is not set, vllm will try to find the nccl library file in the locations specified by LD_LIBRARY_PATH	None	None	None	No SM70 path association
LOCAL_RANK	configuration	local rank of the process in the distributed setting, used to determine the GPU device id	0	0	None	No SM70 path association
MAX_JOBS	tuning	Maximum number of compilation jobs to run in parallel. By default this is the number of CPUs	None	None	None	No SM70 path association
MOONCAKE_PREFERRED_SEGMENT	configuration	Pin this rank to a specific owner segment ("host:port").	None	None	None	No SM70 path association
MOONCAKE_REQUESTER_LOCAL_HOSTNAME	configuration	Override the hostname the rank registers as a Mooncake requester.	None	None	None	No SM70 path association
NO_COLOR	configuration	Standard unix flag for disabling ANSI color codes	False	False	None	No SM70 path association
NVCC_THREADS	tuning	Number of threads to use for nvcc By default this is 1. If set, MAX_JOBS will be reduced to avoid oversubscribing the CPU.	None	None	None	No SM70 path association
Q_SCALE_CONSTANT	configuration	Divisor for dynamic query scale factor calculation for FP8 KV Cache	200	200	None	No SM70 path association
S3_ACCESS_KEY_ID	configuration	S3 access information, used for tensorizer to load model from S3	None	None	None	No SM70 path association
S3_ENDPOINT_URL	configuration	Override the endpoint URL for the S3 model storage connection.	None	None	None	No SM70 path association
S3_SECRET_ACCESS_KEY	configuration	Secret key for the S3 model storage connection; values are never included in this reference.	None	None	None	No SM70 path association
VERBOSE	configuration	If set, vllm will print verbose logs during installation	False	False	None	No SM70 path association
VLLM_ALLOW_CHUNKED_LOCAL_ATTN_WITH_HYBRID_KV_CACHE	configuration	Allow chunked local attention with hybrid kv cache manager. Currently using the Hybrid KV cache manager with chunked local attention in the Llama4 models (the only models currently using chunked local attn) causes a latency regression. For this reason, we disable it by default. This flag is used to allow users to enable it if they want to (to save on kv-cache memory usage and enable longer contexts) TODO(lucas): Remove this flag once latency regression is resolved.	True	True	None	No SM70 path association
VLLM_ALLOW_INSECURE_SERIALIZATION	configuration	If set, allow insecure serialization using pickle. This is useful for environments where it is deemed safe to use the insecure method and it is needed for some reason.	False	False	None	No SM70 path association
VLLM_ALLOW_LONG_MAX_MODEL_LEN	tuning	If the env var VLLM_ALLOW_LONG_MAX_MODEL_LEN is set, it allows the user to specify a max sequence length greater than the max length derived from the model's config.json. To enable this, set VLLM_ALLOW_LONG_MAX_MODEL_LEN=1.	False	False	None	No SM70 path association
VLLM_ALLOW_RUNTIME_LORA_UPDATING	configuration	If set, allow loading or unloading lora adapters in runtime,	False	False	None	No SM70 path association
VLLM_ALLREDUCE_USE_FLASHINFER	configuration	Whether to use FlashInfer allreduce	False	False	None	No SM70 path association
VLLM_ALLREDUCE_USE_SYMM_MEM	configuration	Whether to use pytorch symmetric memory for allreduce	True	True	None	No SM70 path association
VLLM_API_KEY	configuration	API key for vLLM API server	None	None	None	No SM70 path association
VLLM_ASSETS_CACHE	configuration	Path to the cache for storing downloaded assets	os.path.join(VLLM_CACHE_ROOT, 'assets')	Computed when unset: os.path.expanduser(os.path.join(get_default_cache_root(), 'vllm', 'assets'))	None	No SM70 path association
VLLM_ASSETS_CACHE_MODEL_CLEAN	configuration	If the env var is set, we will clean model file in this path $VLLM_ASSETS_CACHE/model_streamer/$model_name	False	False	None	No SM70 path association
VLLM_AUDIO_FETCH_TIMEOUT	configuration	Timeout for fetching audio when serving multimodal models Default is 10 seconds	10	10	None	No SM70 path association
VLLM_BATCH_INVARIANT	configuration	Enable batch-invariant mode: deterministic results regardless of batch composition. Requires NVIDIA GPU with compute capability >= 9.0.	False	False	None	No SM70 path association
VLLM_BLOCKSCALE_FP8_GEMM_FLASHINFER	tuning	Allow use of FlashInfer FP8 block-scale GEMM for linear layers. This uses TensorRT-LLM kernels and requires SM90+ (Hopper).	True	True	None	No SM70 path association
VLLM_CACHE_ROOT	configuration	================== Runtime Env Vars ================== Root directory for vLLM cache files Defaults to ~/.cache/vllm unless XDG_CACHE_HOME is set	os.path.expanduser('~/.cache/vllm')	Computed when unset: os.path.expanduser(os.path.join(get_default_cache_root(), 'vllm'))	None	No SM70 path association
VLLM_CI_USE_S3	configuration	Whether to use S3 path for model loading in CI via RunAI Streamer	False	False	None	No SM70 path association
VLLM_COMPILE_CACHE_SAVE_FORMAT	configuration	Format for saving torch.compile cache artifacts - "binary": saves as binary file Safe for multiple vllm serve processes accessing the same torch compile cache. - "unpacked": saves as directory structure (for inspection/debugging) NOT multiprocess safe - race conditions may occur with multiple processes. Allows viewing and setting breakpoints in Inductor's code output files.	'binary'	'binary'	None	No SM70 path association
VLLM_COMPUTE_NANS_IN_LOGITS	configuration	Enable checking whether the generated logits contain NaNs, indicating corrupted output. Useful for debugging low level bugs or bad hardware but it may add compute overhead.	False	False	None	No SM70 path association
VLLM_CONFIGURE_LOGGING	configuration	Logging configuration If set to 0, vllm will not configure logging If set to 1, vllm will configure logging using the default configuration or the configuration file specified by VLLM_LOGGING_CONFIG_PATH	True	True	None	No SM70 path association
VLLM_CONFIG_ROOT	configuration	Root directory for vLLM configuration files Defaults to ~/.config/vllm unless XDG_CONFIG_HOME is set Note that this not only affects how vllm finds its configuration files during runtime, but also affects how vllm installs its configuration files during **installation**.	os.path.expanduser('~/.config/vllm')	Computed when unset: os.path.expanduser(os.path.join(get_default_config_root(), 'vllm'))	None	No SM70 path association
VLLM_CPU_ATTN_SPLIT_KV	tuning	(CPU backend only) whether to enable attention spilt KV.	True	True	None	No SM70 path association
VLLM_CPU_CI_ENV	configuration	Select the CPU CI platform emulation environment.	None	None	'0' at vllm/platforms/cpu.py:check_and_update_config	No SM70 path association
VLLM_CPU_INT4_W4A8	configuration	(CPU backend only) whether to use SGLang INT4 W4A8 kernels for AWQ.	True	True	None	No SM70 path association
VLLM_CPU_KVCACHE_SPACE	configuration	(CPU backend only) CPU key-value cache space. default is None and will be set as 4 GB	0	Computed when unset: int('0') if 'VLLM_CPU_KVCACHE_SPACE' in os.environ else None	None	No SM70 path association
VLLM_CPU_NUM_OF_RESERVED_CPU	configuration	(CPU backend only) CPU cores not used by OMP threads . Those CPU cores will not be used by OMP threads of a rank.	None	Computed when unset: int('0') if 'VLLM_CPU_NUM_OF_RESERVED_CPU' in os.environ else None	None	No SM70 path association
VLLM_CPU_OMP_THREADS_BIND	tuning	(CPU backend only) CPU core ids bound by OpenMP threads, e.g., "0-31", "0,1,2", "0-31,33". CPU cores of different ranks are separated by '|'.	'auto'	'auto'	None	No SM70 path association
VLLM_CPU_SGL_KERNEL	configuration	(CPU backend only) whether to use SGL kernels, optimized for small batch.	False	False	None	No SM70 path association
VLLM_CPU_SIM_MULTI_NUMA	configuration	Simulate multiple NUMA nodes in CPU platform checks.	None	None	'0' at vllm/utils/ompmultiprocessing.py:__init__	No SM70 path association
VLLM_CUDAGRAPH_INPUT_ADDR_DEBUG	debug	Debug CUDA graph replay input tensor address stability without enabling global DEBUG logging.	False	False	None	No SM70 path association
VLLM_CUDART_SO_PATH	configuration	In some system, find_loaded_library() may not work. So we allow users to specify the path through environment variable VLLM_CUDART_SO_PATH.	None	None	None	No SM70 path association
VLLM_CUDA_COMPATIBILITY_PATH	configuration	Path to the CUDA compatibility libraries when CUDA compatibility is enabled.	None	None	None	No SM70 path association
VLLM_CUSTOM_ALLREDUCE_BLOCK_LIMIT	configuration	vLLM: custom allreduce block limit. The consumer locations and unset defaults are listed below.	None	Computed when unset: int(os.environ['VLLM_CUSTOM_ALLREDUCE_BLOCK_LIMIT']) if 'VLLM_CUSTOM_ALLREDUCE_BLOCK_LIMIT' in os.environ else None	None	No SM70 path association
VLLM_CUSTOM_SCOPES_FOR_PROFILING	configuration	Add optional custom scopes for profiling, disable to avoid overheads	False	False	None	No SM70 path association
VLLM_CUTLASS_SRC_DIR	configuration	Override the CUTLASS source directory used by the local build.	None	None	None at vllm/model_executor/layers/sm70_sparse_attention.py:sparse_extension None at vllm/model_executor/layers/sm70_attention.py:flashattn_extension	No SM70 path association
VLLM_DBO_COMM_SMS	configuration	The number of SMs/CUs to allocate for communication kernels when running DBO; the rest will be allocated to compute. Default: 20 on CUDA (SMs), 64 on ROCm (CUs).	20	Computed when unset: int('64' if hasattr(__import__('torch').version, 'hip') and __import__('torch').version.hip is not None else '20')	None	No SM70 path association
VLLM_DEBUG_DUMP_PATH	debug	Dump fx graphs to the given directory. It will override CompilationConfig.debug_dump_path if set.	None	None	None	No SM70 path association
VLLM_DEBUG_LOG_API_SERVER_RESPONSE	debug	Whether to log responses from API Server for debugging	False	False	None	No SM70 path association
VLLM_DEBUG_MFU_METRICS	debug	Debug logging for --enable-mfu-metrics	False	False	None	No SM70 path association
VLLM_DEBUG_WORKSPACE	debug	Debug workspace allocations. logging of workspace resize operations.	False	False	None	No SM70 path association
VLLM_DEEPEPLL_NVFP4_DISPATCH	configuration	Whether to use DeepEPLL kernels for NVFP4 quantization and dispatch method only supported on Blackwell GPUs and with https://github.com/deepseek-ai/DeepEP/pull/341	False	False	None	No SM70 path association
VLLM_DEEPEP_BUFFER_SIZE_MB	configuration	The size in MB of the buffers (NVL and RDMA) used by DeepEP	1024	1024	None	No SM70 path association
VLLM_DEEPEP_HIGH_THROUGHPUT_FORCE_INTRA_NODE	configuration	Force DeepEP to use intranode kernel for inter-node communication in high throughput mode. This is useful archive higher prefill throughput on system supports multi-node nvlink (e.g GB200).	False	False	None	No SM70 path association
VLLM_DEEPEP_LOW_LATENCY_USE_MNNVL	configuration	Allow DeepEP to use MNNVL (multi-node nvlink) for internode_ll kernel, turn this for better latency on GB200 like system	False	False	None	No SM70 path association
VLLM_DEEP_GEMM_WARMUP	configuration	DeepGemm JITs the kernels on-demand. The warmup attempts to make DeepGemm JIT all the required kernels before model execution so there is no JIT'ing in the hot-path. However, this warmup increases the engine startup time by a couple of minutes. Available options: - "skip" : Skip warmup. - "full" : Warmup deepgemm by running all possible gemm shapes the engine could encounter. - "relax" : Select gemm shapes to run based on some heuristics. The heuristic aims to have the same effect as running all possible gemm shapes, but provides no guarantees.	'relax'	'relax'	None	No SM70 path association
VLLM_DISABLED_KERNELS	configuration	List of quantization kernels that should be disabled, used for testing and performance comparisons. Currently only affects MPLinearKernel selection (kernels: MacheteLinearKernel, MarlinLinearKernel, ExllamaLinearKernel)	[]	Computed when unset: [] if 'VLLM_DISABLED_KERNELS' not in os.environ else os.environ['VLLM_DISABLED_KERNELS'].split(',')	None	No SM70 path association
VLLM_DISABLE_COMPILE_CACHE	configuration	vLLM: disable compile cache. The consumer locations and unset defaults are listed below.	False	Computed by disable_compile_cache	None	No SM70 path association
VLLM_DISABLE_LOG_LOGO	configuration	Disable logging of vLLM logo at server startup time.	False	False	None	No SM70 path association
VLLM_DISABLE_PYNCCL	configuration	Disable pynccl (using torch.distributed instead)	False	False	None	No SM70 path association
VLLM_DISABLE_REQUEST_ID_RANDOMIZATION	configuration	Temporary: skip adding random suffix to internal request IDs. May be needed for KV connectors that match request IDs across instances.	False	False	None	No SM70 path association
VLLM_DISABLE_SHARED_EXPERTS_STREAM	configuration	Disables parallel execution of shared_experts via separate cuda stream	False	False	None	No SM70 path association
VLLM_DIST_IDENT	configuration	Identify the distributed instance when deriving communication names.	None	None	required at vllm/distributed/device_communicators/cpu_communicator.py:make_group_name	No SM70 path association
VLLM_DOCKER_BUILD_CONTEXT	configuration	Used to mark that setup.py is running in a Docker build context, in order to force the use of precompiled binaries.	False	False	None	No SM70 path association
VLLM_DO_NOT_TRACK	configuration	vLLM: do not track. The consumer locations and unset defaults are listed below.	False	False	None	No SM70 path association
VLLM_DP_MASTER_IP	configuration	IP address of the master node in the data parallel setting	'127.0.0.1'	'127.0.0.1'	None	No SM70 path association
VLLM_DP_MASTER_PORT	configuration	Port of the master node in the data parallel setting	0	0	None	No SM70 path association
VLLM_DP_RANK	configuration	Rank of the process in the data parallel setting	0	0	None	No SM70 path association
VLLM_DP_RANK_LOCAL	configuration	Rank of the process in the data parallel setting. Defaults to VLLM_DP_RANK when not set.	-1	When unset: int(sys.modules[__name__].VLLM_DP_RANK)	None	No SM70 path association
VLLM_DP_SIZE	configuration	World size of the data parallel setting	1	1	None	No SM70 path association
VLLM_ELASTIC_EP_DRAIN_REQUESTS	configuration	Whether to wait for all requests to drain before sending the scaling command in elastic EP.	False	False	None	No SM70 path association
VLLM_ELASTIC_EP_SCALE_UP_LAUNCH	configuration	Whether it is a scale up launch engine for elastic EP, Should only be set by EngineCoreClient.	False	False	None	No SM70 path association
VLLM_ENABLE_CUDAGRAPH_GC	configuration	Controls garbage collection during CUDA graph capture. If set to 0 (default), enables GC freezing to speed up capture time. If set to 1, allows GC to run during capture.	False	False	None	No SM70 path association
VLLM_ENABLE_CUDA_COMPATIBILITY	configuration	Enable CUDA compatibility mode for datacenter GPUs with older driver versions than the CUDA toolkit major version of vLLM.	False	False	None	No SM70 path association
VLLM_ENABLE_FLA_PACKED_RECURRENT_DECODE	configuration	vLLM: enable fla packed recurrent decode. The consumer locations and unset defaults are listed below.	True	True; configuration may replace the unset default under the automatic conditions below.	vllm/config/vllm.py:__post_init__ sets '1' when sm70_flash_v100_baseline and env_name not in os.environ; automatic defaults apply only when the environment override is absent.	No SM70 path association
VLLM_ENABLE_FUSED_MOE_ACTIVATION_CHUNKING	configuration	vLLM: enable fused moe activation chunking. The consumer locations and unset defaults are listed below.	False	False	None	No SM70 path association
VLLM_ENABLE_INDUCTOR_COORDINATE_DESCENT_TUNING	configuration	If set to 1, enable coordinate_descent_tuning; By default, this is enabled (1)	True	True	None	No SM70 path association
VLLM_ENABLE_INDUCTOR_MAX_AUTOTUNE	tuning	Enable max_autotune & coordinate_descent_tuning in inductor_config to compile static shapes passed from compile_sizes in compilation_config If set to 1, enable max_autotune; By default, this is enabled (1)	True	True	None	No SM70 path association
VLLM_ENABLE_MAMBA_PREFIX_ASYNC	configuration	Legacy 0.0.3 compatibility knob. Latest vLLM no longer has the old Mamba-prefix async-scheduling hard block; see VllmConfig for the no-op compatibility warning when this is explicitly set.	False	False	None	No SM70 path association
VLLM_ENABLE_MOE_DP_CHUNK	configuration	vLLM: enable moe dp chunk. The consumer locations and unset defaults are listed below.	False	False	None	No SM70 path association
VLLM_ENABLE_PREGRAD_PASSES	configuration	Inductor's pre-grad passes don't do anything for vLLM. The pre-grad passes get run even on cache-hit and negatively impact vllm cold compile times by O(1s) Can remove this after the following issue gets fixed TODO(luka): maybe_inplace requires this https://github.com/pytorch/pytorch/issues/174502	True	True	None	No SM70 path association
VLLM_ENABLE_RESPONSES_API_STORE	configuration	Enables support for the "store" option in the OpenAI Responses API. When set to 1, vLLM's OpenAI server will retain the input and output messages for those requests in memory. By default, this is disabled (0), and the "store" option is ignored. NOTE/WARNING: 1. Messages are kept in memory only (not persisted to disk) and will be lost when the vLLM server shuts down. 2. Enabling this option will cause a memory leak, as stored messages are never removed from memory until the server terminates.	False	False	None	No SM70 path association
VLLM_ENABLE_V1_MULTIPROCESSING	configuration	If set, enable multiprocessing in LLM for the V1 code path.	True	True	None	No SM70 path association
VLLM_ENFORCE_STRICT_TOOL_CALLING	configuration	When 1,the model structural tags will be used to enforce the model output conforming to the model's tool-calling format and schema. Default 0 (off).	False	False	None	No SM70 path association
VLLM_ENGINE_ITERATION_TIMEOUT_S	configuration	timeout for each iteration in the engine	60	60	None	No SM70 path association
VLLM_ENGINE_READY_TIMEOUT_S	configuration	Timeout in seconds for waiting for engine cores to become ready during startup. Default is 600 seconds (10 minutes).	600	600	None	No SM70 path association
VLLM_EXECUTE_MODEL_TIMEOUT_SECONDS	configuration	Timeout in seconds for execute_model RPC calls in multiprocessing executor (only applies when TP > 1).	300	300	None	No SM70 path association
VLLM_FLASHINFER_ALLREDUCE_BACKEND	configuration	Flashinfer fused allreduce backend.	'auto'	'auto'	None	No SM70 path association
VLLM_FLASHINFER_ALLREDUCE_FUSION_THRESHOLDS_MB	tuning	Specifies the thresholds of the communicated tensor sizes under which vllm should use flashinfer fused allreduce. The variable should be a JSON with the following format: { : } Unspecified world sizes will fall back to { 2: 64, 4: 1, : 0.5 }	Computed when unset: json.loads('{}')	Computed when unset: json.loads('{}')	None	No SM70 path association
VLLM_FLASHINFER_AUTOTUNE_CACHE_DIR	configuration	Override the directory for the FlashInfer autotune config cache.	None	None	None	No SM70 path association
VLLM_FLASHINFER_MOE_BACKEND	deprecated	Flashinfer MoE backend for vLLM's fused Mixture-of-Experts support. Both require compute capability 10.0 or above. Available options: - "throughput": [default] Uses CUTLASS kernels optimized for high-throughput batch inference. - "latency": Uses TensorRT-LLM kernels optimized for low-latency inference. Deprecated: pass --moe-backend flashinfer_{trtllm,cutlass,cutedsl} directly.	'latency'	'latency'	None	No SM70 path association
VLLM_FLASHINFER_WORKSPACE_BUFFER_SIZE	configuration	Control the workspace buffer size for the FlashInfer backend.	394 * 1024 * 1024	413138944	None	No SM70 path association
VLLM_FLASH_V100_E4M3_GROUPED_FP32	experimental	Controls grouped E4M3 decode with FP32 accumulation. Default on for supported native layouts to retain qualified numerical behavior. Set 0 to compare the original grouped-attention route during diagnosis.	True	True	None	FlashAttentionV100Backend
VLLM_FLASH_V100_PREFILL_D256_GQA_V37	configuration	Selects the legacy v37 D256/GQA prefill implementation. Default off because the qualified Q8000 FP32 route is preferred. Set 1 only for a matched rollback or a prefill regression investigation.	False	False	None	FlashAttentionV100Backend
VLLM_FLOAT32_MATMUL_PRECISION	configuration	Controls PyTorch float32 matmul precision mode within vLLM workers. Valid options mirror torch.set_float32_matmul_precision	'highest'	'highest'	None	No SM70 path association
VLLM_FORCE_AOT_LOAD	configuration	Force vllm to always load AOT compiled models from disk. Failure to load will result in a hard error when this is enabled. Will be ignored when VLLM_USE_AOT_COMPILE is disabled.	False	False	None	No SM70 path association
VLLM_FUN_AUDIOCHAT_DEBUG	debug	Legacy FunAudioChat multimodal embedding diagnostics from 0.0.3. Default-off; when enabled it prints tensor shapes/norms and can dump one-sample audio embeddings for offline quality debugging.	False	False	None	No SM70 path association
VLLM_FUN_AUDIOCHAT_DUMP_PATH	debug	vLLM: fun audiochat dump path. The consumer locations and unset defaults are listed below.	''	''	None	No SM70 path association
VLLM_FUSED_MOE_CHUNK_SIZE	tuning	Legacy 0.0.3 FusedMoE activation chunk controls. Latest keeps this explicit/default-off until model-level memory, route, and quality gates prove the chunked path for the target backend.	16 * 1024	Computed when unset: max(1, int(str(16 * 1024)))	None	No SM70 path association
VLLM_GC_DEBUG	debug	GC debug config - VLLM_GC_DEBUG=0: disable GC debugger - VLLM_GC_DEBUG=1: enable GC debugger with gc.collect elpased times - VLLM_GC_DEBUG='{"top_objects":5}': enable GC debugger with top 5 collected objects	''	''	None	No SM70 path association
VLLM_GLM53_PP_MHC_MATERIALIZE	configuration	vLLM: glm53 pp mhc materialize. The consumer locations and unset defaults are listed below.	False	False	None	No SM70 path association
VLLM_GPT_OSS_HARMONY_SYSTEM_INSTRUCTIONS	configuration	Allows harmony instructions to be injected on system messages	False	False	None	No SM70 path association
VLLM_GPT_OSS_SYSTEM_TOOL_MCP_LABELS	configuration	Valid values are container,code_interpreter,web_search_preview ex VLLM_GPT_OSS_SYSTEM_TOOL_MCP_LABELS=container,code_interpreter If the server_label of your mcp tool is not in this list it will be completely ignored.	set()	Computed by env_set_with_choices('VLLM_GPT_OSS_SYSTEM_TOOL_MCP_LABELS', default=[], choices=['container', 'code_interpreter', 'web_search_preview'])	None	No SM70 path association
VLLM_GPU_NIC_PCIE_MAPPING	configuration	Comma-separated GPU_BDF=NIC_BDF pairs for RDMA NIC selection. Must be set together with VLLM_NIC_SELECTION_VARS.	''	''	None	No SM70 path association
VLLM_HAS_FLASHINFER_CUBIN	configuration	If set, it means we pre-downloaded cubin files and flashinfer will read the cubin files directly.	False	False	None	No SM70 path association
VLLM_HOST_IP	configuration	used in distributed environment to determine the ip address of the current node, when the node has multiple network interfaces. If you are using multi-node inference, you should set this differently on each node.	''	''	None	No SM70 path association
VLLM_HTTP_TIMEOUT_KEEP_ALIVE	configuration	Timeout in seconds for keeping HTTP connections alive in API server	5	5	None	No SM70 path association
VLLM_HUMMING_INPUT_QUANT_CONFIG	configuration	The activation dtype config for humming kernel	None	Computed when unset: maybe_convert_json_str_or_file(None)	None	No SM70 path association
VLLM_HUMMING_MOE_GEMM_TYPE	configuration	Whether to use indexed gemm for humming moe if 1, force use indexed gemm if 0, force use grouped gemm if None, choose better gemm type automatically	None	None	None	No SM70 path association
VLLM_HUMMING_ONLINE_QUANT_CONFIG	configuration	The online quantization dtype for humming kernel	None	Computed when unset: maybe_convert_json_str_or_file(None)	None	No SM70 path association
VLLM_HUMMING_USE_F16_ACCUM	configuration	Whether to use fp16 accumulator mma	False	Computed when unset: maybe_convert_bool('0')	None	No SM70 path association
VLLM_IMAGE_FETCH_TIMEOUT	configuration	Timeout for fetching images when serving multimodal models Default is 5 seconds	5	5	None	No SM70 path association
VLLM_KEEP_ALIVE_ON_ENGINE_DEATH	configuration	If set, the OpenAI API server will stay alive even after the underlying AsyncLLMEngine errors and stops serving requests	False	False	None	No SM70 path association
VLLM_KPOOL_SKIP_DECODE_WRITE	configuration	vLLM: kpool skip decode write. The consumer locations and unset defaults are listed below.	None	None	None at vllm/model_executor/layers/sparse_attn_indexer_kpool.py:sparse_attn_indexer_kpool	No SM70 path association
VLLM_KPOOL_SKIP_TAIL_CACHE	configuration	vLLM: kpool skip tail cache. The consumer locations and unset defaults are listed below.	None	None	None at vllm/model_executor/layers/sparse_attn_indexer_kpool.py:sparse_attn_indexer_kpool	No SM70 path association
VLLM_KV_CACHE_LAYOUT	configuration	KV Cache layout used throughout vllm. Some common values are: - NHD - HND Where N=num_blocks, H=num_heads and D=head_size. The default value will leave the layout choice to the backend. Mind that backends may only implement and support a subset of all possible layouts.	None	None	None	No SM70 path association
VLLM_KV_EVENTS_USE_INT_BLOCK_HASHES	configuration	Represent block hashes in KV cache events as 64-bit integers instead of raw bytes. Defaults to True for backward compatibility.	True	True	None	No SM70 path association
VLLM_LOGGING_COLOR	configuration	Controls colored logging output. Options: "auto" (default, colors when terminal), "1" (always use colors), "0" (never use colors)	'auto'	'auto'	None	No SM70 path association
VLLM_LOGGING_CONFIG_PATH	configuration	vLLM: logging config path. The consumer locations and unset defaults are listed below.	None	None	None	No SM70 path association
VLLM_LOGGING_LEVEL	configuration	this is used for configuring the default logging level	'INFO'	'INFO'	None	No SM70 path association
VLLM_LOGGING_PREFIX	configuration	if set, VLLM_LOGGING_PREFIX will be prepended to all log messages	''	''	None	No SM70 path association
VLLM_LOGGING_STREAM	configuration	this is used for configuring the default logging stream	'ext://sys.stdout'	'ext://sys.stdout'	None	No SM70 path association
VLLM_LOG_BATCHSIZE_INTERVAL	configuration	vLLM: log batchsize interval. The consumer locations and unset defaults are listed below.	-1	-1.0	None	No SM70 path association
VLLM_LOG_MODEL_INSPECTION	configuration	Log model inspection after loading. If enabled, logs a transformers-style hierarchical view of the model with quantization methods and attention backends.	False	False	None	No SM70 path association
VLLM_LOG_STATS_INTERVAL	configuration	If set, vllm will log stats at this interval in seconds If not set, vllm will log stats every 10 seconds.	10.0	10.0	None	No SM70 path association
VLLM_LOOPBACK_IP	configuration	Used to force set up loopback IP	''	''	None	No SM70 path association
VLLM_LORA_DISABLE_PDL	configuration	Disable PDL for LoRA, as enabling PDL with LoRA on SM100 causes Triton compilation to fail.	False	False	None	No SM70 path association
VLLM_LORA_ENABLE_DUAL_STREAM	configuration	Whether to enable dual cuda streams for LoRA computation (used by both BaseLinearLayerWithLoRA and FusedMoEWithLoRA to overlap the base layer compute with the LoRA fast path).	False	False	None	No SM70 path association
VLLM_LORA_RESOLVER_CACHE_DIR	configuration	a local directory to look in for unrecognized LoRA adapters. only works if plugins are enabled and VLLM_ALLOW_RUNTIME_LORA_UPDATING is enabled.	None	None	None	No SM70 path association
VLLM_LORA_RESOLVER_HF_REPO_LIST	configuration	A remote HF repo(s) containing one or more LoRA adapters, which may be downloaded and leveraged as needed. Only works if plugins are enabled and VLLM_ALLOW_RUNTIME_LORA_UPDATING is enabled. Values should be comma separated.	None	None	None	No SM70 path association
VLLM_MAIN_CUDA_VERSION	configuration	Main CUDA version of vLLM. This follows PyTorch but can be overridden.	'13.0'	'13.0'	None	No SM70 path association
VLLM_MAMBA_ALIGN_CPU_POSTPROCESS	configuration	vLLM: mamba align cpu postprocess. The consumer locations and unset defaults are listed below.	False	False	None	No SM70 path association
VLLM_MAMBA_ALIGN_DEBUG	debug	Legacy 0.0.3 Mamba/GDN alignment diagnostics. Default-off because it logs per-request state movement and accepted-token correction data.	False	False	None	No SM70 path association
VLLM_MARLIN_INPUT_DTYPE	configuration	The activation dtype for marlin kernel	None	None	None	No SM70 path association
VLLM_MARLIN_USE_ATOMIC_ADD	configuration	Whether to use atomicAdd reduce in gptq/awq marlin kernel.	False	False	None	No SM70 path association
VLLM_MAX_AUDIO_CLIP_FILESIZE_MB	tuning	Maximum filesize in MB for a single audio file when processing speech-to-text requests. Files larger than this will be rejected. Default is 25 MB	25	25	None	No SM70 path association
VLLM_MAX_N_SEQUENCES	tuning	Maximum allowed value for the n sampling parameter (number of output sequences per request). Limits resource consumption to prevent denial-of-service via excessively large fan-out. Default: 16384.	16384	16384	None	No SM70 path association
VLLM_MAX_TOKENS_PER_EXPERT_FP4_MOE	tuning	Control the maximum number of tokens per expert supported by the NVFP4 MoE CUTLASS Kernel. This value is used to create a buffer for the blockscale tensor of activations NVFP4 Quantization. This is used to prevent the kernel from running out of memory.	163840	163840	None	No SM70 path association
VLLM_MEDIA_CACHE	configuration	Directory for caching media downloads (images, video, audio fetched from URLs during inference). Empty string disables caching.	''	''	None	No SM70 path association
VLLM_MEDIA_CACHE_MAX_SIZE_MB	tuning	Maximum cache size in MB. When exceeded, least-recently-used entries are evicted. Default is 5120 (5 GB).	5120	5120	None	No SM70 path association
VLLM_MEDIA_CACHE_TTL_HOURS	configuration	Time-to-live in hours for cached media files. Entries older than this are evicted regardless of cache size. Default is 24 hours.	24	24.0	None	No SM70 path association
VLLM_MEDIA_CONNECTOR	configuration	Media connector implementation. - "http": Default connector that supports fetching media via HTTP. Custom implementations can be registered via @MEDIA_CONNECTOR_REGISTRY.register("my_custom_media_connector") and imported at runtime. If a non-existing backend is used, an AssertionError will be thrown.	'http'	'http'	None	No SM70 path association
VLLM_MEDIA_FETCH_MAX_RETRIES	tuning	Maximum number of retries for fetching media (images, audio, video) from URLs. Each retry quadruples the timeout. Default is 3.	3	3	None	No SM70 path association
VLLM_MEDIA_LOADING_THREAD_COUNT	configuration	Max number of workers for the thread pool handling media bytes loading. Set to 1 to disable parallel processing. Default is 8	8	8	None	No SM70 path association
VLLM_MEDIA_URL_ALLOW_REDIRECTS	configuration	Whether to allow HTTP redirects when fetching from media URLs. Default to True	True	True	None	No SM70 path association
VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS	debug	If set to 1, enable CUDA graph memory estimation during memory profiling. This profiles CUDA graph memory usage to provide more accurate KV cache memory allocation. Enabled by default as of v0.21.0	True	True; configuration may replace the unset default under the automatic conditions below.	vllm/config/vllm.py:__post_init__ sets '0' when sm70_flash_0dot3_compile_graph and sm70_compile_disabled_by_user and current_platform.is_cuda() and _any_participating_device_is_capability(self, (7, 0)) and envs.VLLM_SM70_FLASH_ATTN_V100 and not self.use_v2_model_runner and 'VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS' not in os.environ; automatic defaults apply only when the environment override is absent.	No SM70 path association
VLLM_MLA_DISABLE	configuration	If set, vLLM will disable the MLA attention optimizations.	False	False	None	No SM70 path association
VLLM_MM_HASHER_ALGORITHM	configuration	Hash algorithm for multimodal content hashing. - "blake3": Default, fast cryptographic hash (not FIPS 140-3 compliant) - "sha256": FIPS 140-3 compliant, widely supported - "sha512": FIPS 140-3 compliant, faster on 64-bit systems Use sha256 or sha512 for FIPS compliance in government/enterprise deployments	'blake3'	'blake3'	None	No SM70 path association
VLLM_MODEL_REDIRECT_PATH	configuration	Use model_redirect to redirect the model name to a local folder. model_redirect can be a json file mapping the model between repo_id and local folder: {"meta-llama/Llama-3.2-1B": "/tmp/Llama-3.2-1B"} or a space separated values table file: meta-llama/Llama-3.2-1B /tmp/Llama-3.2-1B	None	None	None	No SM70 path association
VLLM_MOE_DP_CHUNK_SIZE	tuning	Legacy 0.0.3 DP all-to-all MoE chunk controls. The latest MoE runner has been refactored around custom-op entry points, so the old FusedMoE.forward_impl_chunked loop is not restored here. Keep these visible as pending compatibility knobs until a latest-runner chunk loop is ported and validated.	256	256	None	No SM70 path association
VLLM_MOE_ROUTING_SIMULATION_STRATEGY	configuration	MoE routing strategy selector. See RoutingSimulator.get_available_strategies() # for available strategies. Custom routing strategies can be registered by RoutingSimulator.register_strategy() Note: custom strategies may not produce correct model outputs	''	''	None	No SM70 path association
VLLM_MOE_USE_DEEP_GEMM	configuration	Allow use of DeepGemm specifically for MoE fused ops (overrides only MoE).	True	True	None	No SM70 path association
VLLM_MOONCAKE_ABORT_REQUEST_TIMEOUT	configuration	Timeout (in seconds) for MooncakeConnector in PD disaggregated setup.	480	480	None	No SM70 path association
VLLM_MOONCAKE_BOOTSTRAP_PORT	configuration	Port used for Mooncake handshake between remote agents.	8998	8998	None	No SM70 path association
VLLM_MOONCAKE_DISK_STAGING_USABLE_RATIO	configuration	Fraction of the owner's DirectIO staging buffer to fill per GET batch.	0.9	0.9	None	No SM70 path association
VLLM_MOONCAKE_STORE_TIER_LOG	configuration	Log per-batch memory/disk tier breakdown on external GETs.	False	False	None	No SM70 path association
VLLM_MQ_BROADCASTER_MAX_CHUNKS	tuning	vLLM: mq broadcaster max chunks. The consumer locations and unset defaults are listed below.	6	6; configuration may replace the unset default under the automatic conditions below.	vllm/config/vllm.py:__post_init__ sets '64' when sm70_flash_0dot3_compile_graph and sm70_compile_disabled_by_user and current_platform.is_cuda() and _any_participating_device_is_capability(self, (7, 0)) and envs.VLLM_SM70_FLASH_ATTN_V100 and 'VLLM_MQ_BROADCASTER_MAX_CHUNKS' not in os.environ; automatic defaults apply only when the environment override is absent.	No SM70 path association
VLLM_MQ_MAX_CHUNKS	tuning	Shared-memory message queue chunk counts. Defaults preserve upstream behavior; increasing these can absorb compile/warmup bursts where worker readers do not consume scheduler broadcasts for a long time.	10	10	None	No SM70 path association
VLLM_MQ_MAX_CHUNK_BYTES_MB	tuning	Control the max chunk bytes (in MB) for the rpc message queue. Object larger than this threshold will be broadcast to worker processes via zmq.	16	16	None	No SM70 path association
VLLM_MSGPACK_ZERO_COPY_THRESHOLD	tuning	Control the threshold for msgspec to use 'zero copy' for serialization/deserialization of tensors. Tensors below this limit will be encoded into the msgpack buffer, and tensors above will instead be sent via a separate message. While the sending side still actually copies the tensor in all cases, on the receiving side, tensors above this limit will actually be zero-copy decoded.	256	256	None	No SM70 path association
VLLM_MULTI_STREAM_GEMM_TOKEN_THRESHOLD	tuning	Token-count cutoff for multi-stream overlap of the attention input GEMM with auxiliary GEMMs (e.g. fused_wqa_wkv overlapped with indexer weights / kv-score projections in DeepSeek-V4). At or below this many tokens the FP8 main GEMM has idle SMs to share with the bf16 aux GEMMs and overlap is a 5-45% win; above it the FP8 GEMM saturates the device and the cross-stream sync becomes pure overhead. Set to 0 to disable the multi-stream path entirely. See #PR 41526 for the empirical result for the default value of 1024 tokens.	1024	1024	None	No SM70 path association
VLLM_MXFP4_USE_MARLIN	deprecated	Whether to use marlin kernel in mxfp4 quantization method Deprecated: use --moe-backend marlin (MoE) or --linear-backend marlin (linear) instead.	None	Computed by deprecated_env('VLLM_MXFP4_USE_MARLIN', 'v0.23', 'Use --moe-backend marlin or --linear-backend marlin.', lambda: maybe_convert_bool(os.environ.get('VLLM_MXFP4_USE_MARLIN', None)))	None	No SM70 path association
VLLM_NCCL_INCLUDE_PATH	configuration	NCCL header path	None	None	None	No SM70 path association
VLLM_NCCL_SO_PATH	configuration	Path to the NCCL library file. It is needed because nccl>=2.19 brought by PyTorch contains a bug: https://github.com/NVIDIA/nccl/issues/1234	None	None	None	No SM70 path association
VLLM_NIC_SELECTION_VARS	configuration	Comma-separated list of env vars to set from the GPU-NIC mapping. Each entry is VAR_NAME or VAR_NAME: (suffix appended to RDMA device name). Must be set together with VLLM_GPU_NIC_PCIE_MAPPING.	''	''	None	No SM70 path association
VLLM_NIXL_EP_MAX_NUM_RANKS	tuning	NIXL EP environment variables	32	32	None	No SM70 path association
VLLM_NIXL_SIDE_CHANNEL_HOST	configuration	IP address used for NIXL handshake between remote agents.	'localhost'	'localhost'	None	No SM70 path association
VLLM_NIXL_SIDE_CHANNEL_PORT	configuration	Port used for NIXL handshake between remote agents.	5600	5600	None	No SM70 path association
VLLM_NO_USAGE_STATS	configuration	vLLM: no usage stats. The consumer locations and unset defaults are listed below.	False	False	None	No SM70 path association
VLLM_NVFP4_GEMM_BACKEND	deprecated	Supported options: - "flashinfer-cudnn": use flashinfer cudnn GEMM backend - "flashinfer-trtllm": use flashinfer trtllm GEMM backend - "flashinfer-cutlass": use flashinfer cutlass GEMM backend - "marlin": use marlin GEMM backend (for GPUs without native FP4 support) - "emulation": use BF16/FP16 GEMM, dequantizing weights and running QDQ on activations. This is only meant for research purposes to run on devices where NVFP4 GEMM kernels are not available. - : automatically pick an available backend Deprecated: use --linear-backend instead.	None	None	None	No SM70 path association
VLLM_NVTX_SCOPES_FOR_PROFILING	debug	Add optional nvtx scopes for profiling, disable to avoid overheads	False	False	None	No SM70 path association
VLLM_OBJECT_STORAGE_SHM_BUFFER_NAME	configuration	Name of the shared memory buffer used for object storage. Only effective when mm_config.mm_processor_cache_type == "shm". Automatically generates a unique UUID-based name per process tree if not explicitly set.	'VLLM_OBJECT_STORAGE_SHM_BUFFER'	Computed by get_env_or_set_default('VLLM_OBJECT_STORAGE_SHM_BUFFER_NAME', lambda: f'VLLM_OBJECT_STORAGE_SHM_BUFFER_{uuid.uuid4().hex}')	None	No SM70 path association
VLLM_PATTERN_MATCH_DEBUG	debug	Debug pattern matching inside custom passes. Should be set to the fx.Node name (e.g. 'getitem_34' or 'scaled_mm_3').	None	None	None	No SM70 path association
VLLM_PLE_CPU_OFFLOAD	configuration	Run n-gram PLE lookup in a dedicated CPU offload worker. The initial implementation supports ModelRunner V1/V2 and node-local MP DP/TP.	False	False; configuration may replace the unset default under the automatic conditions below.	vllm/config/vllm.py:_apply_sm70_qwen38_hybrid_ple_defaults sets '1' when owner policy; automatic defaults apply only when the environment override is absent.	No SM70 path association
VLLM_PLE_DISK_OFFLOAD	configuration	Retain Qwen4Exp PLE safetensor shards as file-backed mappings instead of copying the complete learned n-gram table into anonymous host memory.	False	False; configuration may replace the unset default under the automatic conditions below.	vllm/config/vllm.py:_apply_sm70_qwen38_hybrid_ple_defaults sets '1' when owner policy; automatic defaults apply only when the environment override is absent.	No SM70 path association
VLLM_PLE_DISK_OFFLOAD_NUM_THREADS	tuning	Number of cross-shard mmap gather workers. Zero selects a bounded hardware-aware default.	0	0	None	No SM70 path association
VLLM_PLE_DISK_OFFLOAD_PROFILE	debug	vLLM: ple disk offload profile. The consumer locations and unset defaults are listed below.	False	False	None	No SM70 path association
VLLM_PLE_OFFLOAD_AUTO_NUMA	configuration	Keep the latency-critical PLE lookup process on the NUMA node local to its first visible GPU. This changes CPU placement only; allocations use a local-first policy with fallback so large tables are not forced into a single NUMA node and swapped out.	True	True	None	No SM70 path association
VLLM_PLE_OFFLOAD_PREFAULT	configuration	Fault PLE table pages back into RAM after GPU workers finish loading. Concurrent checkpoint loading can otherwise leave anonymous table pages in swap while reclaimable checkpoint page cache occupies host memory.	True	True	None	No SM70 path association
VLLM_PLE_OFFLOAD_READY_TIMEOUT	configuration	Timeout for PLE weight loading and TP worker registration.	600.0	600.0	None	No SM70 path association
VLLM_PLUGINS	configuration	a list of plugin names to load, separated by commas. if this is not set, it means all plugins will be loaded if this is set to an empty string, no plugins will be loaded	None	Computed when unset: None if 'VLLM_PLUGINS' not in os.environ else os.environ['VLLM_PLUGINS'].split(',')	None	No SM70 path association
VLLM_PORT	configuration	used in distributed environment to manually set the communication port Note: if VLLM_PORT is set, and some code asks for multiple ports, the VLLM_PORT will be used as the first port, and the rest will be generated by incrementing the VLLM_PORT value.	None	Computed by get_vllm_port	None	No SM70 path association
VLLM_PP_LAYER_PARTITION	configuration	Pipeline stage partition strategy	None	None	None	No SM70 path association
VLLM_PROCESS_NAME_PREFIX	configuration	Used to set the process name prefix for vLLM processes. This is useful for debugging and monitoring purposes. The default value is "VLLM".	'VLLM'	'VLLM'	None	No SM70 path association
VLLM_QWEN3NEXT_ENABLE_SHARED_MOE_OVERLAP	configuration	Allows shared-expert stream overlap on eligible Qwen3Next/Qwen4Exp layers. Default enabled by the Qwen4Exp policy, otherwise the standalone getter is off. Set 0 to diagnose stream ordering; placement and DBO guards still decide eligibility.	False	False; configuration may replace the unset default under the automatic conditions below.	vllm/config/vllm.py:_apply_sm70_qwen38_decode_defaults sets '1' when name not in os.environ; automatic defaults apply only when the environment override is absent.	SM70 runtime/kernel policy
VLLM_QWEN3_NEXT_FUSED_SIGMOID_GATING	configuration	Legacy 0.0.3 coarse Qwen3Next fused-sigmoid gate. Latest splits this into recurrent schedule and mixed-QKV controls; keep the old name visible for command compatibility and route-ledger diagnostics.	True	True	None	No SM70 path association
VLLM_QWEN4EXP_PLE_HOST_GIB	configuration	Sets pinned-host PLE memory in GiB per TP rank. Default automatic from actual table size and available host memory, avoiding a checkpoint-specific manual budget. Set an explicit limit when sharing host RAM or reproducing a fixed-memory deployment.	None	None	None	SM70 runtime/kernel policy
VLLM_QWEN4EXP_PLE_HOST_RESERVE_GIB	configuration	Host memory in GiB the automatic PLE placement leaves untouched for the engine processes, checkpoint loading and other tenants; the rest is shared equally by the tensor-parallel ranks that pin the table. Unset: 25 % of the physical host memory.	None	None	None	No SM70 path association
VLLM_QWEN4EXP_PLE_VRAM_RESERVE_GIB	configuration	Device memory in GiB the automatic PLE placement keeps free for the activation peak and the CUDA graph pool. Unset: 8 % of the device, at most 4 GiB.	None	None	None	No SM70 path association
VLLM_RANDOMIZE_DP_DUMMY_INPUTS	configuration	Randomize inputs during dummy runs when using Data Parallel	False	False	None	No SM70 path association
VLLM_RAY_BUNDLE_INDICES	configuration	Bundle indices for Ray, if it is set, it can control precisely which indices are used for the Ray bundle, for every worker. Format: comma-separated list of integers, e.g. "0,1,2,3"	''	''	None	No SM70 path association
VLLM_RAY_DP_PACK_STRATEGY	configuration	Strategy to pack the data parallel ranks for Ray. Available options: - "fill": for DP master node, allocate exactly data-parallel-size-local DP ranks, for non-master nodes, allocate as many DP ranks as can fit; - "strict": allocate exactly data-parallel-size-local DP ranks to each picked node; - "span": Should be used only when a single DP rank requires multiple nodes. allocate one DP rank over as many nodes as required for set world_size; This environment variable is ignored if data-parallel-backend is not Ray.	'strict'	'strict'	None	No SM70 path association
VLLM_RAY_EXTRA_ENV_VARS_TO_COPY	configuration	Comma-separated *additional* individual env var names to copy from the driver to Ray workers. Merged with the built-in defaults defined in vllm.ray.ray\_env (PYTHONHASHSEED). Example: "MY_SECRET,MY_FLAG"	''	''	None	No SM70 path association
VLLM_RAY_EXTRA_ENV_VAR_PREFIXES_TO_COPY	configuration	Comma-separated *additional* prefixes of env vars to copy from the driver to Ray workers. These are merged with the built-in defaults defined in vllm.ray.ray\_env (VLLM_, etc.). Example: "MYLIB_,OTHER_"	''	''	None	No SM70 path association
VLLM_RAY_PER_WORKER_GPUS	configuration	If set, vLLM will pick up the provided Flash Attention MLA Number of GPUs per worker in Ray, if it is set to be a fraction, it allows ray to schedule multiple actors on a single GPU, so that users can colocate other actors on the same GPUs as vLLM.	1.0	1.0	None	No SM70 path association
VLLM_RINGBUFFER_WARNING_INTERVAL	configuration	Interval in seconds to log a warning message when the ring buffer is full	60	60	None	No SM70 path association
VLLM_ROCM_AITER_MOE_DISPATCH_POLICY	configuration	MoE sorting dispatch policy for AITER fused MoE kernels. 0 = auto (default): single-pass for small batches, multi-pass for large batches 1 = always single-pass: one kernel launch, no workspace, may be preferred for low-concurrency decode workloads 2 = always multi-pass: can be faster for MoE-heavy models (e.g., +2-5% on Qwen3-Next, +1.5% on DeepSeek-V3 at TP4, see PR #39177 for benchmarks)	0	0	None	No SM70 path association
VLLM_ROCM_CUSTOM_PAGED_ATTN	configuration	Legacy ROCm opt-out for the custom paged attention selector. Default-on preserves latest upstream behavior; setting it false restores the old escape hatch for ROCm numerical or backend issues.	True	True	None	No SM70 path association
VLLM_ROCM_FP8_MFMA_PAGE_ATTN	configuration	If set, use the fp8 mfma in rocm paged attention.	False	False	None	No SM70 path association
VLLM_ROCM_FP8_PADDING	configuration	Pad the fp8 weights to 256 bytes for ROCm	True	True	None	No SM70 path association
VLLM_ROCM_MOE_PADDING	configuration	Pad the weights for the moe kernel	True	True	None	No SM70 path association
VLLM_ROCM_QUICK_REDUCE_CAST_BF16_TO_FP16	configuration	Custom quick allreduce kernel for MI3* cards Due to the lack of the bfloat16 asm instruction, bfloat16 kernels are slower than fp16, If environment variable is set to 1, the input is converted to fp16	True	True	None	No SM70 path association
VLLM_ROCM_QUICK_REDUCE_MAX_SIZE_BYTES_MB	tuning	Custom quick allreduce kernel for MI3* cards. Controls the maximum allowed number of data bytes(MB) for custom quick allreduce communication. Default: 2048 MB. Data exceeding this size will use either custom allreduce or RCCL communication.	None	Computed when unset: maybe_convert_int(None)	None	No SM70 path association
VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB	tuning	Custom quick allreduce kernel for MI3* cards. Controls the minimum allowed number of data bytes(MB) required to use custom quick allreduce communication. If unset, use the built-in threshold table.	None	Computed when unset: maybe_convert_int(None)	None	No SM70 path association
VLLM_ROCM_QUICK_REDUCE_QUANTIZATION	configuration	Custom quick allreduce kernel for MI3* cards Choice of quantization level: FP, INT8, INT6, INT4 or NONE Recommended for large models to get allreduce	'NONE'	'NONE'	None	No SM70 path association
VLLM_ROCM_QUICK_REDUCE_QUANTIZATION_MIN_SIZE_KB	tuning	Controls the minimum tensor size (KB, where 1 KB = 1024 bytes) required to use the configured QuickReduce codec. Smaller tensors use FP QuickReduce. This does not affect QuickReduce eligibility.	None	Computed when unset: maybe_convert_int(None)	None	No SM70 path association
VLLM_ROCM_SHUFFLE_KV_CACHE_LAYOUT	configuration	Whether to use the shuffled kv cache layout	False	False	None	No SM70 path association
VLLM_ROCM_SLEEP_MEM_CHUNK_SIZE	tuning	flag to control the chunk size (in MB) for sleeping memory allocations under ROCm	256	256	None	No SM70 path association
VLLM_ROCM_USE_AITER	configuration	Disable aiter ops unless specifically enabled. Acts as a parent switch to enable the rest of the other operations.	False	False	None	No SM70 path association
VLLM_ROCM_USE_AITER_FP4BMM	configuration	Whether to use aiter triton fp4 bmm kernel By default is enabled.	True	True	None	No SM70 path association
VLLM_ROCM_USE_AITER_FP4_ASM_GEMM	configuration	Whether to use aiter fp4 gemm asm. By default is disabled.	False	False	None	No SM70 path association
VLLM_ROCM_USE_AITER_FP8BMM	configuration	Whether to use aiter triton fp8 bmm kernel By default is enabled.	True	True	None	No SM70 path association
VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS	configuration	Whether to use aiter fusion shared experts ops. By default is disabled.	False	False	None	No SM70 path association
VLLM_ROCM_USE_AITER_LINEAR	configuration	use aiter linear op if aiter ops are enabled The following list of related ops - scaled_mm (per-tensor / rowwise) - use aiter tuned gemms for unquantized gemms	True	True	None	No SM70 path association
VLLM_ROCM_USE_AITER_MHA	configuration	Whether to use aiter mha ops. By default is enabled.	True	True	None	No SM70 path association
VLLM_ROCM_USE_AITER_MLA	configuration	Whether to use aiter mla ops. By default is enabled.	True	True	None	No SM70 path association
VLLM_ROCM_USE_AITER_MOE	configuration	Whether to use aiter moe ops. By default is enabled.	True	True	None	No SM70 path association
VLLM_ROCM_USE_AITER_PAGED_ATTN	configuration	Whether to use aiter paged attention. By default is disabled.	False	False	None	No SM70 path association
VLLM_ROCM_USE_AITER_RMSNORM	configuration	use aiter rms norm op if aiter ops are enabled.	True	True	None	No SM70 path association
VLLM_ROCM_USE_AITER_TRITON_GEMM	configuration	Whether to use aiter triton kernels for gemm ops. By default is enabled.	True	True	None	No SM70 path association
VLLM_ROCM_USE_AITER_TRITON_ROPE	configuration	Whether to use aiter rope. By default is disabled.	False	False	None	No SM70 path association
VLLM_ROCM_USE_AITER_UNIFIED_ATTENTION	configuration	Use AITER triton unified attention for V1 attention	False	False	None	No SM70 path association
VLLM_ROCM_USE_SKINNY_GEMM	configuration	use rocm skinny gemms	True	True	None	No SM70 path association
VLLM_RPC_BASE_PATH	configuration	path used for ipc when the frontend api server is running in multi-processing mode to communicate with the backend engine process.	tempfile.gettempdir()	Computed when unset: tempfile.gettempdir()	None	No SM70 path association
VLLM_RPC_TIMEOUT	configuration	Time in ms for the zmq client to wait for a response from the backend server for simple data operations	10000	10000	None	No SM70 path association
VLLM_RUST_FRONTEND_PATH	configuration	Path to the Rust frontend binary. Defaults to "auto" which discovers the binary installed with the vllm package. Only used when VLLM_USE_RUST_FRONTEND=1.	'auto'	Computed when unset: _resolve_rust_frontend_path()	None	No SM70 path association
VLLM_SERVER_DEV_MODE	configuration	If set, vllm will run in development mode, which will enable some additional endpoints for developing and debugging, e.g. /reset_prefix_cache	False	False	None	No SM70 path association
VLLM_SHARED_EXPERTS_STREAM_TOKEN_THRESHOLD	tuning	Limits when we run shared_experts in a separate stream. We found out that for large batch sizes, the separate stream execution is not beneficial (most likely because of the input clone) TODO(alexm-redhat): Tune to be more dynamic based on GPU type	256	256	None	No SM70 path association
VLLM_SKIP_MODEL_NAME_VALIDATION	configuration	Skip model name validation in OpenAI API requests. When set to 1, any model name will be accepted in the 'model' field of API requests. This is useful for proxy/gateway scenarios where the actual model is served but different names may be used in requests.	False	False	None	No SM70 path association
VLLM_SKIP_P2P_CHECK	configuration	We assume drivers can report p2p status correctly. If the program hangs when using custom allreduce, potantially caused by a bug in the driver (535 series), if might be helpful to set VLLM_SKIP_P2P_CHECK=0 so that vLLM can verify if p2p is actually working. See https://github.com/vllm-project/vllm/blob/a9b15c606fea67a072416ea0ea115261a2756058/vllm/distributed/device\_communicators/custom\_all\_reduce\_utils.py#L101-L108 for details. # noqa	False	True	None	No SM70 path association
VLLM_SKIP_PRECOMPILED_VERSION_SUFFIX	configuration	If set, skip adding +precompiled suffix to version string	False	False	None	No SM70 path association
VLLM_SLEEP_WHEN_IDLE	configuration	Legacy 0.0.3 mp broadcast idle-wait knob. Latest upstream replaced the old SpinTimer/SpinSleepTimer split with SpinCondition, which already spins under load and waits on notification while idle. Kept only for command compatibility and inventory closure.	False	False	None	No SM70 path association
VLLM_SM70_DEBUG	debug	Select comma-separated SM70 diagnostics: trace logs, MTP phase timing, decode events, attention routing or eager selector shadow checks (selector). Default empty to avoid logging and observer overhead. Use only to diagnose a route or latency issue; explicit channels override old aliases.	set()	set()	None	SM70 diagnostics
VLLM_SM70_DFLASH2_TAIL_CUDAGRAPHS	configuration	Captures supported single-request DFlash2 verifier tails. Default on after paired token/state and speed qualification. Set 0 to trade tail-graph memory and speed for the eager fallback during diagnosis.	True	True	None	DFlash2 verifier
VLLM_SM70_E4M3_LONG_ATTENTION	configuration	Controls the built-in E4M3 long-context attention route. Default enabled when the native operator is present; dtype and page-layout checks still apply. Set 0 to diagnose attention with the original full-context route.	None	None; unset admits the built-in native operator or an explicit manifest.	vllm/v1/attention/ops/sm70_e4m3_long.py:long_attention_enabled	Flash-V100 long-context attention
VLLM_SM70_FLASH_ATTN_V100	configuration	Selects Flash-V100 attention automatically on V100. Default on to use the qualified SM70 backend. Set 0 only to diagnose backend selection; prefer the engine attention_backend option when choosing a backend.	True	True	None	FlashAttentionV100Backend
VLLM_SM70_FUSED_SIGMOID_MIXED_QKV	configuration	Fuses sigmoid and multiply for supported mixed-QKV layouts without changing FP32 recurrent state. Default on after #703 quality qualification. Set 0 to diagnose the separate epilogue.	True	True; operator dtype, geometry, numerical policy and native availability checks still apply.	None	SM70 runtime/kernel policy, Flash-Next qualified batch
VLLM_SM70_MOE_ADD_ALLREDUCE	experimental	Fuses shared and routed MoE outputs with TP all-reduce. Default enabled by the qualified Qwen4Exp policy, otherwise off. Set 0 to compare separate addition and communication when investigating MoE overlap.	False	False; configuration may replace the unset default under the automatic conditions below.	vllm/config/vllm.py:_apply_sm70_qwen38_decode_defaults sets '1' when name not in os.environ; automatic defaults apply only when the environment override is absent.	SM70 collectives
VLLM_SM70_MTP_HC_BATCH	configuration	Enables qualified M5/M10 HC projection and gather kernels with the existing FP16 split-K boundaries. Default on after #703 qualification. Set 0 to recover HC packed weights when QWEN38_BATCH_FASTPATH is also 0 (330 MiB per Flash-Next TP4 target rank).	True	True; operator dtype, geometry, numerical policy and native availability checks still apply.	None	MTP verifier, Flash-Next qualified batch
VLLM_SM70_MTP_HC_COOPERATIVE	configuration	Combines admitted HC projection and gather phases with cooperative barriers. Default on after #703 quality and complete- round qualification. Set 0 to compare the separate-phase implementation when diagnosing HC performance.	True	True; operator dtype, geometry, numerical policy and native availability checks still apply.	None	MTP verifier, Flash-Next qualified batch
VLLM_SM70_MTP_HC_FULL_UNROLL	configuration	Selects the qualified fully unrolled cooperative HC schedule. Default on after #703 complete-round qualification. Set 0 to compare the partial-unroll schedule when diagnosing performance.	True	True; operator dtype, geometry, numerical policy and native availability checks still apply.	None	MTP verifier, Flash-Next qualified batch
VLLM_SM70_MTP_MOE_FP16_EXACT	configuration	Uses exact native M1/M5 FP16 draft MoE kernels with the original accumulation order. Default on after #703 qualification. Set 0 to compare the original Triton MoE implementation.	True	True; operator dtype, geometry, numerical policy and native availability checks still apply.	None	MTP verifier, Flash-Next qualified batch
VLLM_SM70_MTP_PLE_CONV	configuration	Fuses qualified MTP4 PLE rollback, ordered convolution taps, SiLU and state commit. Default on after #703 exact state and model qualification. Set 0 to diagnose PLE state updates with the separate operations.	True	True; operator dtype, geometry, numerical policy and native availability checks still apply.	None	MTP verifier, Flash-Next qualified batch
VLLM_SM70_MTP_ROUTER_BATCH	configuration	Enables the exact M5/M10 FP16 router projection with four ordered FP32 splits. Default on after #703 qualification. Set 0 to recover packed router copies (122.5 MiB per Flash-Next TP4 target rank).	True	True; operator dtype, geometry, numerical policy and native availability checks still apply.	None	MTP verifier, Flash-Next qualified batch
VLLM_SM70_MTP_ROUTER_TOP16	configuration	Selects lossless top-16 router keys before the unchanged top-10 normalization. Default on after #703 qualification. Set 0 to diagnose routing with the original full-key selection.	True	True; operator dtype, geometry, numerical policy and native availability checks still apply.	None	MTP verifier, Flash-Next qualified batch
VLLM_SM70_MTP_SHARED_BATCH	configuration	Enables exact M5/M10 shared-expert projection and SiLU fusion. Default on after #703 qualification. Set 0 to recover packed shared-expert copies (76.5625 MiB per Flash-Next TP4 target rank).	True	True; operator dtype, geometry, numerical policy and native availability checks still apply.	None	MTP verifier, Flash-Next qualified batch
VLLM_SM70_MTP_SPLIT_DRAFT_CUDAGRAPHS	configuration	Separates MTP draft and verifier graph sizes so single-token draft graphs remain available. Default enabled by the qualified Qwen4Exp MTP engine policy, otherwise off. Set 0 only to diagnose graph capture; speculative width is configured through speculative_config.	False	False	None	MTP verifier
VLLM_SM70_NVFP4_MOE_GROUPED_MTP5	configuration	Reuses expert grouping for the qualified NVFP4 MTP5 route without changing W13 split order. Default on after #703 quality and complete-round qualification. Set 0 to compare the direct expert dispatch.	True	True; operator dtype, geometry, numerical policy and native availability checks still apply.	None	TurboMind NVFP4, Flash-Next qualified batch
VLLM_SM70_QSA_MTP_TOPK	configuration	Enables the qualified M5/M10 QSA top-k compaction while retaining the original scorer. Default on after #703 qualification. Set 0 to compare full-row top-k when diagnosing selector output.	True	True; operator dtype, geometry, numerical policy and native availability checks still apply.	None	QSA sparse attention/indexer, Flash-Next qualified batch
VLLM_SM70_QWEN38_BATCH_FASTPATH	configuration	Enables the qualified small-batch dense, GDN and TP4 HC routes. Default on after #704 quality and throughput qualification. Set 0 to trade speed for KV capacity; Flash-Next TP4 packed copies add up to 1055.625 MiB per rank.	True	True; operator dtype, geometry, numerical policy and native availability checks still apply.	None	SM70 runtime/kernel policy, Flash-Next qualified batch
VLLM_SM70_QWEN38_FP16_GEMV	configuration	Selects checkpoint-FP16 SM70 row GEMV for supported Qwen4Exp projections. Default enabled by the engine policy after capability admission; the standalone getter is off. Set 0 to compare the ordinary linear method when investigating a projection regression.	False	False; configuration may replace the unset default under the automatic conditions below.	vllm/config/vllm.py:_apply_sm70_qwen38_decode_defaults sets '1' when name not in os.environ; automatic defaults apply only when the environment override is absent.	SM70 FP16 linear
VLLM_SM70_QWEN38_FUSED_GDN_INPUT_FP16	experimental	Fuses supported checkpoint-FP16 GDN input projections and stores. Default enabled by the qualified Qwen4Exp policy, otherwise off. Set 0 to isolate the original input projection chain during diagnosis.	False	False; configuration may replace the unset default under the automatic conditions below.	vllm/config/vllm.py:_apply_sm70_qwen38_decode_defaults sets '1' when name not in os.environ; automatic defaults apply only when the environment override is absent.	Gated DeltaNet
VLLM_SM70_QWEN38_FUSED_HC_FP16	experimental	Fuses supported checkpoint-FP16 HyperConnection stages. Default enabled by the Qwen4Exp policy, otherwise off; local geometry and collective guards remain. Set 0 to isolate HC projection and mixing during diagnosis.	False	False; configuration may replace the unset default under the automatic conditions below.	vllm/config/vllm.py:_apply_sm70_qwen38_decode_defaults sets '1' when name not in os.environ; automatic defaults apply only when the environment override is absent.	SM70 FP16 linear
VLLM_SM70_QWEN38_GDN_INPUT_BATCH	configuration	Automatic packing is qualified for ordinary decode and MTP; other proposers retain explicit opt-in. Enables lossless packed GDN input projections for M2..16. Default on after #703/#704 qualification. Set 0 together with QWEN38_BATCH_FASTPATH=0 to recover the packed-weight memory (725.625 MiB per Flash-Next TP4 target rank).	True	True; operator dtype, geometry, numerical policy and native availability checks still apply.	None	Gated DeltaNet, Flash-Next qualified batch
VLLM_SM70_QWEN38_HYBRID_PLE	configuration	Keeps local pinned-host PLE shards for decode while prefill uses CPU/disk offload. Default enabled by the eligible Qwen4Exp engine policy, otherwise off. Set 0 only to compare PLE placement when diagnosing host-memory or transfer behavior.	False	False; configuration may replace the unset default under the automatic conditions below.	vllm/config/vllm.py:_apply_sm70_qwen38_hybrid_ple_defaults sets '1' when owner policy; automatic defaults apply only when the environment override is absent.	SM70 runtime/kernel policy
VLLM_SM70_QWEN4_EXP_ONLINE_QPN8	experimental	Requantizes selected FP16 checkpoint weights to channel E4M3 for online QPN8. Default off because this changes weight precision. Set 1 only for an explicitly qualified precision/performance experiment; keep it off for checkpoint-preserving inference.	False	False	None	SM70 runtime/kernel policy
VLLM_SM70_REQUIRE_PROFILE_ACCELERATION	debug	Requires the release profile acceleration selfcheck to pass at startup. Default off so other models and diagnostic fallbacks can start. Set 1 when validating the declared release profile; it does not certify that requests hit every kernel.	False	False	None	SM70 runtime/kernel policy
VLLM_SM70_RMSNORM_GATED_EXACT	configuration	Uses native FP32 gated RMSNorm for supported SM70 layouts. Default on after #703/#704 quality qualification to preserve arithmetic across compiled graph contexts. Other models remain off until qualified. Set 0 only to diagnose the original normalization route.	False	False at the compatibility getter; resolved per engine to True for qualified Flash-Next ordinary/MTP configurations. Operator dtype, geometry, numerical policy and native availability checks still apply.	Flash-Next ordinary decode or MTP quality qualification	SM70 runtime/kernel policy, Flash-Next qualified batch
VLLM_SM70_TP4_PUSH_ALLREDUCE	configuration	Controls the qualified SM70 TP4 push all-reduce. Default on for supported topology, payload and graph layouts; per-engine GLM policy may disable it. Set 0 to isolate collective transport when investigating correctness or latency.	True	True; configuration may replace the unset default under the automatic conditions below.	vllm/config/vllm.py:_configure_sm70_glm5_dflash_tp4_push_allreduce sets '0' when env_name not in os.environ; automatic defaults apply only when the environment override is absent.	SM70 collectives
VLLM_SPARSE_INDEXER_MAX_LOGITS_MB	tuning	Maximum size (in MB) for logits tensor in sparse MLA indexer prefill chunks. Bounds the [M, N] float32 logits tensor to prevent CUDA OOM. Default: 512 MB	512	512	None	No SM70 path association
VLLM_SPEC_DEBUG_CORRUPTION	debug	Speculative-decoding diagnostic: debug corruption. The consumer locations and unset defaults are listed below.	False	False	None	No SM70 path association
VLLM_SPEC_DUMP_ALIGNMENT	debug	0.0.3 speculative/MTP diagnostics. These are default-off and must not change sampling or model outputs.	False	False	None	No SM70 path association
VLLM_SPEC_DUMP_ALIGNMENT_DIR	debug	Speculative-decoding diagnostic: dump alignment dir. The consumer locations and unset defaults are listed below.	None	None	'/tmp' at vllm/v1/sample/rejection_sampler.py:_maybe_dump_alignment '/tmp' at vllm/v1/worker/gpu/spec_decode/dflash2/sparse_rejection.py:_maybe_dump_selector_alignment	No SM70 path association
VLLM_SPEC_DUMP_ALIGNMENT_LIMIT	debug	Speculative-decoding diagnostic: dump alignment limit. The consumer locations and unset defaults are listed below.	3	3	None	No SM70 path association
VLLM_SPEC_DUMP_ALIGNMENT_STEPS	debug	Speculative-decoding diagnostic: dump alignment steps. The consumer locations and unset defaults are listed below.	None	None	None	No SM70 path association
VLLM_SPEC_DUMP_ALIGNMENT_TAG	debug	Speculative-decoding diagnostic: dump alignment tag. The consumer locations and unset defaults are listed below.	None	None	'' at vllm/v1/sample/rejection_sampler.py:_maybe_dump_alignment '' at vllm/v1/worker/gpu/spec_decode/dflash2/sparse_rejection.py:_maybe_dump_selector_alignment	No SM70 path association
VLLM_SPEC_DUMP_DRAFT_LOGITS	debug	Speculative-decoding diagnostic: dump draft logits. The consumer locations and unset defaults are listed below.	False	False	None	No SM70 path association
VLLM_SSM_CONV_STATE_LAYOUT	configuration	SSM conv state layout used for Mamba models. - SD: (state_len, dim) — dim contiguous (default) - DS: (dim, state_len) — TP-sharded dim on dim1, consistent with SSM temporal state and HND KV cache layout.	None	None	None	No SM70 path association
VLLM_SYSTEM_START_DATE	configuration	Pin the conversation start date injected into the Harmony system message. When unset the current date is used, which introduces non-determinism (different tokens -> different model behaviour at temperature=0). Set to an ISO date string, e.g. "2023-09-12", for reproducible inference or testing.	None	None	None	No SM70 path association
VLLM_TARGET_DEVICE	configuration	================== Installation Time Env Vars ================== Target device of vLLM, supporting [cuda (by default), rocm, cpu]	'cuda'	'cuda'	None	No SM70 path association
VLLM_TEST_FORCE_FP8_MARLIN	configuration	If set, forces FP8 Marlin to be used for FP8 quantization regardless of the hardware support for FP8 compute.	False	False	None	No SM70 path association
VLLM_TEST_FORCE_LOAD_FORMAT	configuration	vLLM: test force load format. The consumer locations and unset defaults are listed below.	'dummy'	'dummy'	None	No SM70 path association
VLLM_TOOL_JSON_ERROR_AUTOMATIC_RETRY	configuration	Enable automatic retry when tool call JSON parsing fails If enabled, returns an error message to the model to retry If disabled (default), raises an exception and fails the request	False	False	None	No SM70 path association
VLLM_TOOL_PARSE_REGEX_TIMEOUT_SECONDS	configuration	Regex timeout for use by the vLLM tool parsing plugins.	1	1	None	No SM70 path association
VLLM_TPU_BUCKET_PADDING_GAP	configuration	Gap between padding buckets for the forward pass. So we have 8, we will run forward pass with [16, 24, 32, ...].	0	Computed when unset: int(os.environ['VLLM_TPU_BUCKET_PADDING_GAP']) if 'VLLM_TPU_BUCKET_PADDING_GAP' in os.environ else 0	None	No SM70 path association
VLLM_TPU_MOST_MODEL_LEN	configuration	vLLM: tpu most model len. The consumer locations and unset defaults are listed below.	None	Computed when unset: maybe_convert_int(None)	None	No SM70 path association
VLLM_TPU_USING_PATHWAYS	configuration	Whether using Pathways	False	False	None	No SM70 path association
VLLM_TP_ALLREDUCE_TRACE	debug	Legacy 0.0.3 diagnostic gate. Logs one TP all-reduce backend decision per group/backend/shape/dtype so route-hit data can distinguish custom AR, pynccl, flashinfer, symmetric-memory, and torch fallback paths.	False	False	None	No SM70 path association
VLLM_TRACE_FUNCTION	debug	Trace function calls If set to 1, vllm will trace function calls Useful for debugging	0	0	None	No SM70 path association
VLLM_TRITON_ATTN_USE_TD	configuration	Use tensor descriptors for Q/K/V loads and output stores in the Triton unified-attention kernel. Enables HW 2D block reads on Intel Xe2/Xe3; the non-TD branch is dead-code-eliminated at Triton compile time so other platforms see no overhead. Tri-state override: unset (default) lets the triton_attn backend auto-select per platform (currently auto-enabled on XPU only); 1 forces TD on; 0 forces TD off. Useful for A/B benchmarking the TD path.	None	None	None	No SM70 path association
VLLM_TUNED_CONFIG_FOLDER	configuration	User override folder for tuned Triton-kernel configs. Shared by MoE, Mamba SSU, and LoRA. Filenames are distinct so one folder can hold all. Each component first checks this folder, then the configs shipped with vLLM (if any). If no JSON matches, it uses a hard-coded heuristic.	None	None	None	No SM70 path association
VLLM_USAGE_SOURCE	configuration	vLLM: usage source. The consumer locations and unset defaults are listed below.	'production'	'production'	None	No SM70 path association
VLLM_USAGE_STATS_SERVER	configuration	Usage stats collection	'https://stats.vllm.ai'	'https://stats.vllm.ai'	None	No SM70 path association
VLLM_USE_AOT_COMPILE	configuration	Feature flag to enable/disable AOT compilation. This will ensure compilation is done in warmup phase and the compilation will be reused in subsequent calls.	False	Computed by use_aot_compile; configuration may replace the unset default under the automatic conditions below.	vllm/config/vllm.py:_configure_sm70_glm5_dflash_tp8_pp1_verifier_path sets '0' when name not in os.environ; automatic defaults apply only when the environment override is absent. vllm/config/vllm.py:__post_init__ sets '1' when sm70_flash_0dot3_compile_graph and sm70_compile_disabled_by_user and current_platform.is_cuda() and _any_participating_device_is_capability(self, (7, 0)) and envs.VLLM_SM70_FLASH_ATTN_V100 and 'VLLM_USE_AOT_COMPILE' not in os.environ; automatic defaults apply only when the environment override is absent.	No SM70 path association
VLLM_USE_BREAKABLE_CUDAGRAPH	experimental	Experimental: breakable cudagraph does not rely on torch.compile	False	False; configuration may replace the unset default under the automatic conditions below.	vllm/config/vllm.py:__post_init__ sets '1' when self.model_config is not None and 'VLLM_USE_BREAKABLE_CUDAGRAPH' not in os.environ and any((a in ('DeepseekV4ForCausalLM', 'DeepSeekV4MTPModel') for a in self.model_config.architectures)); automatic defaults apply only when the environment override is absent. vllm/config/vllm.py:__post_init__ sets '1' when envs.VLLM_SM70_USE_BREAKABLE_CUDAGRAPH and current_platform.is_cuda() and _any_participating_device_is_capability(self, (7, 0)) and 'VLLM_USE_BREAKABLE_CUDAGRAPH' not in os.environ; automatic defaults apply only when the environment override is absent.	No SM70 path association
VLLM_USE_BYTECODE_HOOK	configuration	Feature flag to enable/disable bytecode in TorchCompileWithNoGuardsWrapper.	True	True	None	No SM70 path association
VLLM_USE_DEEP_GEMM	configuration	Allow use of DeepGemm kernels for fused moe ops.	True	True	None	No SM70 path association
VLLM_USE_DEEP_GEMM_E8M0	configuration	Whether to use E8M0 scaling when DeepGEMM is used on Blackwell GPUs.	True	True	None	No SM70 path association
VLLM_USE_DEEP_GEMM_TMA_ALIGNED_SCALES	configuration	Whether to create TMA-aligned scale tensor when DeepGEMM is used.	True	True	None	No SM70 path association
VLLM_USE_EXPERIMENTAL_PARSER_CONTEXT	experimental	Experimental: use this to enable MCP tool calling for non harmony models	False	False	None	No SM70 path association
VLLM_USE_FASTOKENS	configuration	If true, replace the Rust BPE backend that powers HF fast tokenizers with the fastokens (https://github.com/crusoecloud/fastokens) shim. Applies to any tokenizer mode that loads an HF fast tokenizer (hf, deepseek_v32, deepseek_v4, qwen_vl, …). The fastokens Python package must be installed.	False	False	None	No SM70 path association
VLLM_USE_FBGEMM	deprecated	Flag to enable FBGemm kernels on model execution Deprecated: use --linear-backend fbgemm instead.	False	False	None	No SM70 path association
VLLM_USE_FLASHINFER_MOE_FP16	deprecated	Allow use of FlashInfer BF16 MoE kernels for fused moe ops. Deprecated: use --moe-backend to select a kernel explicitly.	False	False	None	No SM70 path association
VLLM_USE_FLASHINFER_MOE_FP4	deprecated	Allow use of FlashInfer NVFP4 MoE kernels for fused moe ops. Deprecated: use --moe-backend to select a kernel explicitly.	False	False	None	No SM70 path association
VLLM_USE_FLASHINFER_MOE_FP8	deprecated	Allow use of FlashInfer FP8 MoE kernels for fused moe ops. Deprecated: use --moe-backend to select a kernel explicitly.	False	False	None	No SM70 path association
VLLM_USE_FLASHINFER_MOE_INT4	configuration	Allow use of FlashInfer MxInt4 MoE kernels for fused moe ops.	False	False	None	No SM70 path association
VLLM_USE_FLASHINFER_MOE_MXFP4_BF16	deprecated	If set to 1, use the FlashInfer BF16 (activation) x MXFP4 (weight) MoE backend. Deprecated: use --moe-backend to select a kernel explicitly.	False	False	None	No SM70 path association
VLLM_USE_FLASHINFER_MOE_MXFP4_MXFP8	deprecated	If set to 1, use the FlashInfer MXFP8 (activation) x MXFP4 (weight) MoE backend. Deprecated: use --moe-backend flashinfer_trtllm combined with --quantization_config.moe.activation mxfp8.	False	False	None	No SM70 path association
VLLM_USE_FLASHINFER_MOE_MXFP4_MXFP8_CUTLASS	deprecated	If set to 1, use the FlashInfer CUTLASS backend for MXFP8 (activation) x MXFP4 (weight) MoE. Deprecated: use --moe-backend flashinfer_cutlass combined with --quantization_config.moe.activation mxfp8.	False	False	None	No SM70 path association
VLLM_USE_FLASHINFER_SAMPLER	configuration	Whether to use the FlashInfer top-k / top-p sampler on CUDA. Enabled by default when the hardware supports it — set to 0 to opt out explicitly, which forces the PyTorch-native (Triton for bs>=8) path.	True	Computed when unset: bool(int(os.environ['VLLM_USE_FLASHINFER_SAMPLER'])) if 'VLLM_USE_FLASHINFER_SAMPLER' in os.environ else True	None	No SM70 path association
VLLM_USE_FUSED_MOE_GROUPED_TOPK	configuration	Whether to use fused grouped_topk used for MoE expert selection.	True	True	None	No SM70 path association
VLLM_USE_LAYERNAME	configuration	If set to "0", disable LayerName opaque type for layer_name parameters in custom ops. Defaults to enabled on torch >= 2.11.	True	True	None	No SM70 path association
VLLM_USE_MEGA_AOT_ARTIFACT	configuration	Enable loading compiled models directly from cached standalone compile artifacts without re-splitting graph modules. This reduces overhead during model loading by using reconstruct_serializable_fn_from_mega_artifact.	False	Computed by use_mega_aot_artifact	None	No SM70 path association
VLLM_USE_MODELSCOPE	configuration	If true, will load models from ModelScope instead of Hugging Face Hub. note that the value is true or false, not numbers	False	False	None	No SM70 path association
VLLM_USE_NCCL_SYMM_MEM	configuration	Flag to enable NCCL symmetric memory allocation and registration	False	False	None	No SM70 path association
VLLM_USE_NVFP4_CT_EMULATIONS	deprecated	Controls whether or not emulations are used for NVFP4 generations on machines < 100 for compressed-tensors models Deprecated: use --linear-backend emulation instead.	False	False	None	No SM70 path association
VLLM_USE_OINK_OPS	configuration	Optional: enable external Oink custom ops (e.g., Blackwell RMSNorm). Disabled by default.	False	False	None	No SM70 path association
VLLM_USE_PRECOMPILED	configuration	If set, vllm will use precompiled native binaries (*.so and vllm-rs).	False	Computed when unset: ''.strip().lower() in ('1', 'true') or bool(None)	None	No SM70 path association
VLLM_USE_PRECOMPILED_RUST	configuration	If set, vllm will use the precompiled Rust frontend binary (vllm-rs).	False	False	None	No SM70 path association
VLLM_USE_RAY_COMPILED_DAG_CHANNEL_TYPE	configuration	If the env var is set, Ray Compiled Graph uses the specified channel type to communicate between workers belonging to different pipeline-parallel stages. Available options: - "auto": use the default channel type - "nccl": use NCCL for communication - "shm": use shared memory and gRPC for communication	'auto'	'auto'	None	No SM70 path association
VLLM_USE_RAY_COMPILED_DAG_OVERLAP_COMM	experimental	If the env var is set, it enables GPU communication overlap (experimental feature) in Ray's Compiled Graph.	False	False	None	No SM70 path association
VLLM_USE_RAY_V2_EXECUTOR_BACKEND	configuration	When True and distributed_executor_backend="ray", use RayExecutorV2 (MQ-based) instead of RayDistributedExecutor (compiled-graph backend).	True	True	None	No SM70 path association
VLLM_USE_RAY_WRAPPED_PP_COMM	configuration	If the env var is set, it uses a Ray Communicator wrapping vLLM's pipeline parallelism communicator to interact with Ray's Compiled Graph. Otherwise, it uses Ray's NCCL communicator.	True	True	None	No SM70 path association
VLLM_USE_RUST_FRONTEND	configuration	If set, use the Rust frontend binary instead of the Python API server process(es).	False	False	None	No SM70 path association
VLLM_USE_SIMPLE_KV_OFFLOAD	configuration	Enable simple KV offload.	False	False	None	No SM70 path association
VLLM_USE_SPINLOOP_EXT	configuration	If set to 1, use Python spinloop extension to poll in a more efficient way when using the mp backend.	False	False	None	No SM70 path association
VLLM_USE_STANDALONE_COMPILE	configuration	Feature flag to enable/disable Inductor standalone compile. In torch <= 2.7 we ignore this flag; in torch >= 2.9 this is enabled by default.	True	True	None	No SM70 path association
VLLM_USE_TRITON_AWQ	configuration	If set, vLLM will use Triton implementations of AWQ.	False	False	None	No SM70 path association
VLLM_USE_V1	configuration	Legacy benchmark harness knob from 0.0.3. Latest vLLM routes vllm.engine.llm_engine directly to the V1 engine, so this remains a no-op compatibility value.	True	True	None	No SM70 path association
VLLM_USE_V2_MODEL_RUNNER	configuration	Flag to control the v2 model runner. If unset, use config defaults.	None	Computed when unset: maybe_convert_bool(None)	None	No SM70 path association
VLLM_V1_OUTPUT_PROC_CHUNK_SIZE	tuning	Controls the maximum number of requests to handle in a single asyncio task when processing per-token outputs in the V1 AsyncLLM interface. It is applicable when handling a high concurrency of streaming requests. Setting this too high can result in a higher variance of inter-message latencies. Setting it too low can negatively impact TTFT and overall throughput.	128	128	None	No SM70 path association
VLLM_V1_USE_OUTLINES_CACHE	configuration	Whether to turn on the outlines cache for V1 This cache is unbounded and on disk, so it's not safe to use in an environment with potentially malicious users.	False	False	None	No SM70 path association
VLLM_V2_CUDAGRAPH_MEM_MIB	configuration	Explicit CUDA graph memory reservation for the V2 GPU model runner, which does not profile graph memory yet. The default preserves upstream behavior. Set this from a measured capture result when sizing a tight KV cache pool.	0.0	0.0	None	No SM70 path association
VLLM_VIDEO_FETCH_TIMEOUT	configuration	Timeout for fetching videos when serving multimodal models Default is 30 seconds	30	30	None	No SM70 path association
VLLM_VIDEO_LOADER_BACKEND	configuration	Backend for Video IO — selects the frame-sampling algorithm. - "opencv": uniform sampling. - "opencv_dynamic": duration-aware dynamic sampling. - "identity": returns raw video bytes for model processor to handle. Custom backend implementations can be registered via @VIDEO_LOADER_REGISTRY.register("my_custom_video_loader") and imported at runtime. If a non-existing backend is used, an AssertionError will be thrown.	'opencv'	'opencv'	None	No SM70 path association
VLLM_WEIGHT_OFFLOADING_DISABLE_PIN_MEMORY	configuration	Disable using pytorch's pin memory for CPU offloading.	False	False	None	No SM70 path association
VLLM_WEIGHT_OFFLOADING_DISABLE_UVA	configuration	Disable using UVA (Unified Virtual Addressing) for CPU offloading.	False	False	None	No SM70 path association
VLLM_WORKER_MULTIPROC_METHOD	configuration	Use dedicated multiprocess context for workers. Both spawn and fork work	'fork'	'fork'	None	No SM70 path association
VLLM_XGRAMMAR_CACHE_MB	configuration	Control the cache sized used by the xgrammar compiler. The default of 512 MB should be enough for roughly 1000 JSON schemas. It can be changed with this variable if needed for some reason.	512	512	None	No SM70 path association
VLLM_XLA_CACHE_PATH	configuration	Path to the XLA persistent cache directory. Only used for XLA devices such as TPUs.	os.path.join(VLLM_CACHE_ROOT, 'xla_cache')	Computed when unset: os.path.expanduser(os.path.join(get_default_cache_root(), 'vllm', 'xla_cache'))	None	No SM70 path association
VLLM_XLA_CHECK_RECOMPILATION	configuration	If set, assert on XLA recompilation after each execution step.	False	False	None	No SM70 path association
VLLM_XLA_USE_SPMD	configuration	Enable SPMD mode for TPU backend.	False	False	None	No SM70 path association
VLLM_XPU_ENABLE_XPU_GRAPH	configuration	Whether enable XPU graph on Intel GPU	False	False	None	No SM70 path association
VLLM_XPU_USE_SAMPLER_KERNEL	configuration	whether use xpu specific sample kernel	True	True	None	No SM70 path association
VLLM_ZENTORCH_WEIGHT_PREPACK	configuration	(Zen CPU backend) eagerly prepack weights into ZenDNN blocked layout at model load time. Eliminates per-inference layout conversion overhead.	True	True	None	No SM70 path association
V_SCALE_CONSTANT	configuration	Divisor for dynamic value scale factor calculation for FP8 KV Cache	100	100	None	No SM70 path association