1Cat-vLLM

0 Fork 0

Files

T

Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>

2025-05-25 06:05:38 -07:00

auto_awq.md

…

bitblas.md

…

bnb.md

2025-05-25 06:05:38 -07:00

fp8.md

…

gguf.md

…

gptqmodel.md

…

int4.md

…

int8.md

…

modelopt.md

…

quantized_kvcache.md

…

quark.md

…

README.md

…

supported_hardware.md

…

torchao.md

…

title

title
Quantization

{ #quantization-index }

Quantization trades off model precision for smaller memory footprint, allowing large models to be run on a wider range of devices.

Contents: