1Cat-vLLM

chart-helm

Add example of helm chart for vllm deployment on k8s (#9199 )

2024-12-10 09:19:27 +00:00

fp8

…

production_monitoring

…

api_client.py

…

aqlm_example.py

…

cpu_offload.py

…

disaggregated_prefill.sh

[Core] Implement disagg prefill by StatelessProcessGroup (#10502 )

2024-12-01 19:01:00 -06:00

florence2_inference.py

…

gguf_inference.py

…

gradio_openai_chatbot_webserver.py

…

gradio_webserver.py

…

llm_engine_example.py

…

logging_configuration.md

[Core] Fix broken log configuration (#10458 )

2024-11-23 10:23:51 +08:00

lora_with_quantization_inference.py

…

multilora_inference.py

…

offline_chat_with_tools.py

…

offline_inference_arctic.py

…

offline_inference_audio_language.py

…

offline_inference_chat.py

…

offline_inference_cli.py

[Minor] Revert change in offline inference example (#10545 )

2024-11-21 21:28:16 +00:00

offline_inference_distributed.py

…

offline_inference_embedding.py

[Misc] Split up pooling tasks (#10820 )

2024-12-11 01:28:00 -08:00

offline_inference_encoder_decoder.py

…

offline_inference_mlpspeculator.py

…

offline_inference_neuron_int8_quantization.py

…

offline_inference_neuron.py

…

offline_inference_openai.md

…

offline_inference_pixtral.py

…

offline_inference_structured_outputs.py

[Doc] Add documentation for Structured Outputs (#9943 )

2024-11-18 09:52:12 -08:00

offline_inference_tpu.py

…

offline_inference_vision_language_embedding.py

[Misc] Split up pooling tasks (#10820 )

2024-12-11 01:28:00 -08:00

offline_inference_vision_language_multi_image.py

[Doc] Explicitly state that InternVL 2.5 is supported (#10978 )

2024-12-07 16:58:02 +00:00

offline_inference_vision_language.py

PaliGemma 2 support (#11142 )

2024-12-13 07:40:07 +00:00

offline_inference_with_prefix.py

…

offline_inference_with_profiler.py

[V1] Fix torch profiling for offline inference (#11125 )

2024-12-12 15:51:53 +00:00

offline_inference.py

[Feature] vLLM ARM Enablement for AARCH64 CPUs (#9228 )

2024-11-25 18:32:39 -08:00

offline_profile.py

…

openai_chat_completion_client_for_multimodal.py

…

openai_chat_completion_client_with_tools.py

…

openai_chat_completion_client.py

…

openai_chat_completion_structured_outputs.py

[Doc] Add documentation for Structured Outputs (#9943 )

2024-11-18 09:52:12 -08:00

openai_chat_embedding_client_for_multimodal.py

…

openai_completion_client.py

…

openai_cross_encoder_score.py

Support Cross encoder models (#10400 )

2024-11-24 18:56:20 -08:00

openai_embedding_client.py

…

openai_example_batch.jsonl

…

run_cluster.sh

…

save_sharded_state.py

…

template_alpaca.jinja

…

template_baichuan.jinja

…

template_blip2.jinja

…

template_chatglm2.jinja

…

template_chatglm.jinja

…

template_chatml.jinja

…

template_dse_qwen2_vl.jinja

…

template_falcon_180b.jinja

…

template_falcon.jinja

…

template_inkbot.jinja

…

template_llava.jinja

…

template_vlm2vec.jinja

…

tensorize_vllm_model.py

…

tool_chat_template_granite_20b_fc.jinja

…

tool_chat_template_granite.jinja

Change granite chat template to keep json list formatting for tool calls (#10452 )

2024-11-19 18:16:54 -07:00

tool_chat_template_hermes.jinja

…

tool_chat_template_internlm2_tool.jinja

…

tool_chat_template_llama3.1_json.jinja

[Bugfix][Frontend] Update Llama Chat Templates to also support Non-Tool use (#10164 )

2024-11-23 10:17:38 +08:00

tool_chat_template_llama3.2_json.jinja

[Misc] Update llama 3.2 template to support system prompt with images (#10901 )

2024-12-05 05:54:06 +00:00

tool_chat_template_llama3.2_pythonic.jinja

[Frontend] Pythonic tool parser (#9859 )

2024-11-14 04:14:34 +00:00

tool_chat_template_mistral_parallel.jinja

…

tool_chat_template_mistral.jinja

…

tool_chat_template_toolace.jinja

[Frontend] Pythonic tool parser (#9859 )

2024-11-14 04:14:34 +00:00