yangzhuxinyzx
b3c9ce45f0
Preserve the sparse-attention extension during precompiled reuse, tag full-ABI native wheels for the build interpreter, and release cancelled video uploads before returning. Assisted-by: OpenAI Codex Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
61 lines
2.2 KiB
Markdown
61 lines
2.2 KiB
Markdown
# Dockerfile
|
|
|
|
We provide a [docker/Dockerfile](../../../docker/Dockerfile) to construct the image for running an OpenAI compatible server with vLLM.
|
|
More information about deploying with Docker can be found [here](../../deployment/docker.md).
|
|
|
|
SM70 builds include CPython-specific extensions and produce wheels for the
|
|
build interpreter (for example, `cp312-cp312` for Python 3.12). `setup.py`
|
|
automatically overrides `--py-limited-api=cp38` for these wheels, including
|
|
when the final stage reuses precompiled extensions. Build with the same Python
|
|
minor version as the intended runtime. Builds containing only stable-ABI
|
|
extensions retain the requested `abi3` tag.
|
|
|
|
Below is a visual representation of the multi-stage Dockerfile. The build graph contains the following nodes:
|
|
|
|
- All build stages
|
|
- The default build target (highlighted in grey)
|
|
- External images (with dashed borders)
|
|
|
|
The edges of the build graph represent:
|
|
|
|
- `FROM ...` dependencies (with a solid line and a full arrow head)
|
|
|
|
- `COPY --from=...` dependencies (with a dashed line and an empty arrow head)
|
|
|
|
- `RUN --mount=(.\*)from=...` dependencies (with a dotted line and an empty diamond arrow head)
|
|
|
|
> <figure markdown="span">
|
|
> { align="center" alt="query" width="100%" }
|
|
> </figure>
|
|
>
|
|
> Made using: <https://github.com/patrickhoefler/dockerfilegraph>
|
|
>
|
|
> Commands to regenerate the build graph (make sure to run it **from the \`root\` directory of the vLLM repository** where the dockerfile is present):
|
|
>
|
|
> ```bash
|
|
> dockerfilegraph \
|
|
> -o png \
|
|
> --legend \
|
|
> --dpi 200 \
|
|
> --max-label-length 50 \
|
|
> --filename docker/Dockerfile
|
|
> ```
|
|
>
|
|
> or in case you want to run it directly with the docker image:
|
|
>
|
|
> ```bash
|
|
> docker run \
|
|
> --rm \
|
|
> --user "$(id -u):$(id -g)" \
|
|
> --workdir /workspace \
|
|
> --volume "$(pwd)":/workspace \
|
|
> ghcr.io/patrickhoefler/dockerfilegraph:alpine \
|
|
> --output png \
|
|
> --dpi 200 \
|
|
> --max-label-length 50 \
|
|
> --filename docker/Dockerfile \
|
|
> --legend
|
|
> ```
|
|
>
|
|
> (To run it for a different file, you can pass in a different argument to the flag `--filename`.)
|