GHSA-85xf-c7hm-whqw vLLM: Structured-output request errors escape the request boundary and terminate the shared EngineCore — engine-fatal denial of service (3 sites)

• PyPI vllm: < 0.30.0 → fixed in 0.30.0

Affected • Ecosystem / package: pip / `vllm` • Affected versions: vLLM ≤ 0.25.1 (confirmed on 0.25.1, commit `752a3a504485`). The lower bound predates 0.25.1; maintainers can confirm how far back each path reaches. Summary Three structured-output request paths let an ordinary request reach a condition that raises an uncaught, engine-fatal exception instead of a per-request validation error. The failure is not confined to the request that caused it: it escapes into `EngineCore`'s busy loop and triggers a fatal `_send_engine_dead()`, so one malformed structured-output request denies service to all concurrent and subsequent tenants of that engine. The requests are ordinary API calls; the client does not need any privileged configuration beyond the (often default) structured-output feature. The shared root cause is the absence of a per-request exception boundary around structured-output grammar/token handling: a value that should fail as a request-scoped validation error instead propagates as an uncaught exception (or bypasses frontend validation entirely) and reaches the scheduler/engine loop, which treats the failure as fatal. These sites are distinct from the published fixes for GHSA-6qc9-v4r8-22xg and GHSA-8wr5-jm2h-8r4f: GHSA-6qc9 (PR #17623) added frontend json-schema/regex/type validation but the completion check on v0.25.1 still catches only `TimeoutError`, so Site 1's valid duplicate-root EBNF via the latched xgrammar backend still re-raises and kills EngineCore; GHSA-8wr5 (PR #44744) fixes recovered-token state in the eagle spec-decode path and explicitly does not treat trailing `-1` padding as the fault, so Site 2's `-1` reaching `backend_guidance.validate_tokens()` via `ngram_gpu` survives it. Affected code Links pinned to the confirmed commit `752a3a504485` (v0.25.1). The three sites are: Site 1 — latched-backend compile exception escapes. `StructuredOutputManager` permanently latches one backend on the first structured-output request and always compiles through it, ignoring the per-request `auto` backend selection. A grammar that `auto` accepts only via a fallback backend (e.g. a duplicate-root EBNF) then makes the latched xgrammar compiler raise; the exception is stored in the grammar `Future`, and the completion check catches only `TimeoutError`, so `Future.result()` re-raises during scheduler promotion and kills EngineCore. • Backend defaults to `"auto"`: `vllm/config/structured_outputs.py#L21`. • The `auto` validator tries xgrammar and silently falls back, recording the resolved backend on the request: `vllm/sampling_params.py#L1004-L1035` (fields at `#L85-L88`). • The manager latches one backend and always compiles through it: `vllm/v1/structured_output/__init__.py#L127-L159` and `_create_grammar()` at `#L173-L184`. • The xgrammar sink calls the native compiler unguarded: `vllm/v1/structured_output/backend_xgrammar.py#L78-L110` (grammar path at `#L90`). • The completion check catches only `TimeoutError`: `vllm/v1/structured_output/request.py#L48-L63` (`result(timeout=0.0001)` at `#L55`, `except TimeoutError` at `#L57`). • Scheduler promotion dereferences the grammar with no exception boundary: `vllm/v1/core/sched/scheduler.py#L2441-L2460` (reached from `schedule()` at #L396). • The uncaught exception is treated as fatal: `vllm/v1/engine/core.py#L1229-L1234` (`_send_engine_dead()` at `#L1470`). The escaping-exception sink: the completion check only handles `TimeoutError`, so any compile exception stored in the `Future` re-raises out of `result()` during scheduler promotion. ```python vllm/v1/structured_output/request.py Lines 48-59 def _check_grammar_completion(self) -> bool: # NOTE: We have to lazy import to gate circular imports from vllm.v1.request import RequestStatus if isinstance(self._grammar, Future): try: # We will check whether the future is ready within 100 us self._grammar = self._grammar.result(timeout=0.0001) self.status = RequestStatus.WAITING except TimeoutError: return False return True ``` Site 2 — guidance treats `ngram_gpu` `-1` padding as a token id. The `ngram_gpu` proposer pads a fixed-width draft-token row with `-1` and separately records `num_valid_draft_tokens`, but the scheduler forwards the untrimmed padded row into `GuidanceGrammar.validate_tokens()`, which passes `-1` to the native `llguidance` matcher; the matcher cannot convert a negative value to its unsigned token type and raises an `OverflowError`, uncaught and fatal. • The proposer fills with `-1` and computes `num_valid_draft_tokens`: `vllm/v1/spec_decode/ngram_proposer_gpu.py#L189-L209`. • `_get_draft_token_ids_cpu()` materializes the full padded row, not trimmed to the valid count: `vllm/v1/worker/gpu_model_runner.py#L4835-L4849`. • Two scheduler call sites pass the padded row into `validate_tokens(...)`: `vllm/v1/core/sched/scheduler.py#L1967` and `#L1997`. • The sink has no negative-value filter: `vllm/v1/structured_output/backend_guidance.py#L181-L192` (`validate_tokens(tokens)` at [`#L192`](https://github.com/vllm-project/vllm/blob/752a3a504485790a2e8491cacbb35c1373…