youkaichaoandMichael Goin 1c27d25fb5 [core][model] yet another cpu offload implementation (#6496) ...

Co-authored-by: Michael Goin <michael@neuralmagic.com>

2024-07-17 20:54:35 -07:00

..

fp8

[mypy] Enable type checking for test directory (#5017)

2024-06-15 04:45:31 +00:00

production_monitoring

[Doc][CI/Build] Update docs and tests to use vllm serve (#6431)

2024-07-17 07:43:21 +00:00

api_client.py

[Doc][CI/Build] Update docs and tests to use vllm serve (#6431)

2024-07-17 07:43:21 +00:00

aqlm_example.py

[Frontend] Add FlexibleArgumentParser to support both underscore and dash in names (#5718)

2024-06-20 17:00:13 -06:00

cpu_offload.py

[core][model] yet another cpu offload implementation (#6496)

2024-07-17 20:54:35 -07:00

fuyu_example.py

[Model] Initialize Fuyu-8B support (#3924)

2024-07-14 05:27:14 +00:00

gradio_openai_chatbot_webserver.py

…

gradio_webserver.py

…

llava_example.py

[CI/Build] vLLM cache directory for images (#6444)

2024-07-15 23:12:25 -07:00

llava_next_example.py

[vlm] Remove vision language config. (#6089)

2024-07-03 22:14:16 +00:00

llm_engine_example.py

[Frontend] Add FlexibleArgumentParser to support both underscore and dash in names (#5718)

2024-06-20 17:00:13 -06:00

logging_configuration.md

[Doc][CI/Build] Update docs and tests to use vllm serve (#6431)

2024-07-17 07:43:21 +00:00

lora_with_quantization_inference.py

…

multilora_inference.py

…

offline_inference_arctic.py

…

offline_inference_distributed.py

[mypy] Enable type checking for test directory (#5017)

2024-06-15 04:45:31 +00:00

offline_inference_embedding.py

…

offline_inference_mlpspeculator.py

[BugFix] Fix cuda graph for MLPSpeculator (#5875)

2024-06-27 04:12:10 +00:00

offline_inference_neuron.py

Unmark more files as executable (#5962)

2024-06-28 17:34:56 -04:00

offline_inference_openai.md

…

offline_inference_tpu.py

[CI/Build][TPU] Add TPU CI test (#6277)

2024-07-15 14:31:16 -07:00

offline_inference_with_prefix.py

…

offline_inference.py

…

openai_chat_completion_client.py

…

openai_completion_client.py

…

openai_embedding_client.py

…

openai_example_batch.jsonl

…

openai_vision_api_client.py

[Doc][CI/Build] Update docs and tests to use vllm serve (#6431)

2024-07-17 07:43:21 +00:00

paligemma_example.py

[CI/Build] vLLM cache directory for images (#6444)

2024-07-15 23:12:25 -07:00

phi3v_example.py

[CI/Build] vLLM cache directory for images (#6444)

2024-07-15 23:12:25 -07:00

save_sharded_state.py

[Frontend] Add FlexibleArgumentParser to support both underscore and dash in names (#5718)

2024-06-20 17:00:13 -06:00

template_alpaca.jinja

…

template_baichuan.jinja

…

template_chatglm2.jinja

…

template_chatglm.jinja

…

template_chatml.jinja

…

template_falcon_180b.jinja

…

template_falcon.jinja

…

template_inkbot.jinja

…

template_llava.jinja

…

tensorize_vllm_model.py

[Frontend] Add FlexibleArgumentParser to support both underscore and dash in names (#5718)

2024-06-20 17:00:13 -06:00