GHSA-pr7f-p5mw-fc87 vLLM: Incomplete CVE-2025-62164 remediation can be bypassed by concurrent prompt parts

• PyPI vllm: >= 0.21.0, < 0.26.0 → fixed in 0.26.0

Executive Summary The follow-up protection for CVE-2025-62164 is incomplete at vLLM revision `26587f9519e22a5c4549ead7595ad9ca3229c4fd`. It wraps serialized prompt-embedding reconstruction and dense conversion in `torch.sparse.check_sparse_tensor_invariants()`, but PyTorch 2.11.0 implements that context with save/enable/restore operations over process-global state. Two prompt-embedding parts in one `/v1/chat/completions` request are gathered concurrently on the event loop's default executor. When one context exits before the other loads its tensor, it can restore the global flag to `False` while the second part remains inside its guard. In a deterministic run against hash-verified source from the affected revision, the actual target loader rejected an invalid sparse payload as a negative control. The frozen chat tracker then scheduled benign and malicious parts on distinct `asyncio_0` and `asyncio_1` threads. The benign context exited, the malicious loader observed the invariant flag disabled, and `torch.load(weights_only=True)` reconstructed indices `[[10], [10]]` for a declared shape of `[3, 3]`. The run intercepted the target's `to_dense()` call before it operated on the invalid tensor. This primary trigger requires `--enable-prompt-embeds`, which is default-off, but it does not require `renderer_num_workers > 1`, a multimodal model, or `--enable-mm-embeds`. API authentication is optional in the stock server: middleware is installed only when CLI or environment API keys are supplied. The lab proves bypass of the follow-up guard, invalid sparse reconstruction, and guarded-sink reachability. Crash and memory-corruption consequences are conditional on the behavior documented by the published CVE. Background CVE-2025-62164 / GHSA-mrw7-hf4f-83pf concerns client-controlled serialized `prompt_embeds` reaching `torch.load(weights_only=True)` and an invalid sparse tensor reaching `to_dense()`. The advisory attributes memory corruption, denial of service, and potential code execution to that historical unsafe operation. The remediation chronology matters for duplicate handling: • PR #27204, merge commit `58fab50d82838d5014f4a14d991fdb9352c9c84b` on 2025-10-22, introduced the default-off `enable_prompt_embeds` gate. It did not add the sparse-invariant context. • Commit `84e23d103d3483f944780d0d42bcf0993fd27e3a` on 2025-12-15, titled `additional protection for CVE-2025-62164 (#30649)`, added the process-global sparse-invariant context around load, type check, and dense conversion. • Refactor commit `f0a1c8453ad1c664c8a04c83fe545195fcd556eb` on 2026-01-31 moved the guarded loader into `vllm/renderers/embed_utils.py` while preserving the same context. • Chat content-part commit `14043dfecd35dd2f12b4d51eb9fa166184a0ca0f` on 2026-05-01 introduced `prompt_embeds` chat parts and the concurrent one-request schedule described here. This report therefore does not present the malformed sparse payload or `to_dense()` sink as new. It reports a distinct concurrency root cause and trigger: unsynchronized save/enable/restore of the process-global follow-up guard, reachable through the later multi-part chat scheduler. The affected revision pins PyTorch 2.11.0 in `pyproject.toml:10`. Vulnerability Details The target's `safe_load_prompt_embeds` performs the guarded operation in `vllm/renderers/embed_utils.py:16-39`: ```python with torch.sparse.check_sparse_tensor_invariants(): tensor = torch.load( BytesIO(pybase64.b64decode(embed, validate=True)), weights_only=True, map_location=torch.device("cpu"), ) if not isinstance(tensor, torch.Tensor): raise VLLMValidationError(...) tensor = tensor.to_dense() ``` The context is not request-local. With the global flag initially disabled, we can describe the verified interleaving: 1. Benign part A enters, saves `False`, and enables the flag. 2. Malicious part B enters, saves `True`, and leaves the flag enabled. 3. A completes its load and exits, restoring its saved `False` value. 4. B remains lexically inside its context but observes the actual global flag as `False`. 5. B's `torch.load(..., weights_only=True)` reconstructs the malformed sparse tensor. 6. The target reaches `tensor.to_dense()` before later rank, hidden-size, and dtype checks. `weights_only=True` constrains deserialization types; it does not compensate for a sparse invariant check that another request has disabled. The complete stock actor-to-sink chain, traced in the affected source, is: `POST /v1/chat/completions` (`vllm/entrypoints/openai/chat_completion/api_router.py:41-61`) -> `OpenAIServingChat.create_chat_completion` -> `_create_chat_completion` -> `render_chat_request` (`vllm/entrypoints/openai/chat_completion/serving.py:206-280`) -> `OnlineRenderer.render_chat` (`vllm/renderers/online_renderer.py:95-190`) -> `preprocess_chat` (`vllm/renderers/online_renderer.py:335-380`) -> `BaseRenderer.render_chat_async` (`vllm/renderers/base.py:1070-1105`) -> `HfRenderer.render_messages_async` (`vllm/renderers/hf.py:1049-1085`) -> `parse_chat_messages_async` (`vllm/entrypoints/chat_utils.py:1911-1945`) -> content-part `parse_prompt_embeds` and `_load_prompt_embeds_async` (`vllm/entrypoints/chat_utils.py:1099-1120`) -> `AsyncMultiModalItemTracker.resolve_items` (`vllm/entrypoints/chat_utils.py:818-835`) -> `asyncio.gather` of both prompt parts -> `safe_load_prompt_embeds_async` -> `make_async` -> `loop.run_in_executor(executor=None, ...)` (`vllm/utils/async_utils.py:28-45`) -> guarded `torch.load` -> `to_dense()`. The prompt async helper is created without an explicit executor, so it uses the event loop's default executor. This path is separate from the renderer's configurable pool. The deterministic scheduler run observed the two parts on distinct default-executor threads while leaving `renderer_num_workers` at its default of one. `prompt_embeds` bypasses multimodal processing, and the tracker explicitly permits it when `is_multimodal_model=False` (`vllm/entrypoints/chat_utils.py:793-837`). Consequently, the primary trigger needs neither a multimodal model nor `enable_mm_embeds`. The source also states that async wrappers must be thread-safe (`vllm/utils/async_utils.py:28-38`), while a target test acknowledges that the sparse flag is not thread-local and concurrent users can leak state (`tests/renderers/test_sparse_tensor_validation.py:58-61`). Exploitability Analysis The following evidence labels separate what was demonstrated from what remains conditional: | Label | Claim | | --- | --- | | Verified by run | PyTorch 2.11.0 rejects the identical invalid payload through the actual target loader without the race. | | Verified by run | The hash-verified frozen tracker schedules two prompt parts on distinct default-executor threads, races the flag to `False`, reconstructs the invalid sparse tensor, and reaches the target `to_dense()` call while the interception prevents execution. | | Traced in source | A client can supply multiple `prompt_embeds` content parts through the stock `/v1/chat/completions` route and the function chain above. | | Traced in source | `enable_prompt_embeds` defaults to `False` (`vllm/config/model.py:255-260`), so the operator must opt in. `enable_mm_embeds` and non-default renderer workers are not preconditions for this path. | | Traced in source | `api_key` defaults to `None` (`vllm/entrypoints/openai/cli_args.py:264`), and authentication middleware is installed only when a CLI or enviro…