Ported from ReVolta: https://github.com/Leonccaa/ReVolta/commit/0ecebfdc677d4bb513a4a939bbd8253b7847c12a Original vLLM sources, adapted in ReVolta: https://github.com/vllm-project/vllm/pull/47586 Signed-off-by: Leonccaa <166551845+Leonccaa@users.noreply.github.com> Co-authored-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com> Co-authored-by: Codex <codex@openai.com>