GHSA-p4rw-rvv2-7xwr NLTK: Corpus readers follow symlinks outside trusted roots despite pathsec enforcement

• PyPI nltk: < 3.10.3 → fixed in 3.10.3

Summary Several corpus readers still step outside NLTK's symlink-aware trusted-root model. They derive in-root paths from trusted corpus state, convert those paths back into plain strings, and reopen them with built-in `open()` rather than `nltk.pathsec.open()`. Details • Vulnerability type: Path traversal and symlink boundary bypass • Affected component: `nltk.corpus.reader.ipipan`, `nltk.corpus.reader.crubadan`, `nltk.corpus.reader.lin` • Affected versions: Published `3.9.4` and current source `v3.10.0-rc2` both reproduced. • Patched versions: Not yet patched • Root cause: Root-derived paths are reopened with raw `open()` without preserving the trusted-root boundary. `IPIPANCorpusReader` opens `header.xml` derived from `morph.xml`, `CrubadanCorpusReader` opens `table.txt` directly, and `LinThesaurusCorpusReader` opens `simN.lsp` paths returned from its own root helpers. Under `pathsec.ENFORCE=True`, a symlink placed inside the trusted corpus root can point outside the root and still be parsed successfully. It was confirmed parsed outside-root content is returned through public methods such as `channels()`, `domains()`, `categories()`, `langs()`, `crubadan_to_iso()`, `synonyms()`, and `scored_synonyms()`. PoC Preconditions • The application processes attacker-influenced corpora inside a trusted NLTK data root or trusted corpus directory. Steps 1. Create a trusted corpus root and keep `pathsec.ENFORCE=True` with that root allowlisted. 2. Place symlinked reader inputs such as `header.xml`, `table.txt`, or `simN.lsp` inside the root and point them to external files. 3. Instantiate the corresponding corpus reader and call its normal public methods. 4. Observe that parsed outside-root values are returned even though `pathsec.open()` blocks the same symlink targets. Minimal reproducible excerpt ```text {'ipipan': ['LEAK', 'TOPSECRET', 'CLASSIFIED'], 'crubadan': ['LEAK'], 'lin': [('LEAK', 9.5)]} ``` Impact An attacker who can stage corpus files or symlinks under a trusted data root can disclose outside-root content through normal corpus-reader results, defeating the boundary NLTK documents for shared and untrusted-input environments. Remediation Preserve `PathPointer` and `required_root` semantics end to end. Replace direct `open()` calls with `nltk.pathsec.open()` or a reader helper that keeps the trusted-root boundary intact. References • https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/ipipan.py#L162-L192 • https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/crubadan.py#L74-L98 • https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/lin.py#L40-L43 • https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/pathsec.py#L521-L545 --- Fix + full-codebase audit (verified) I swept every raw file open in the corpus readers, not just the three the umbrella named: | Reader | Site | Advisory | Root scoping | |---|---|---|---| | crubadan | table.txt + `-3grams.txt` | p4rw / j5pw | `required_root=self.root` | | lin | simN.lsp | p4rw | `required_root=self.root` | | xmldocs | XMLCorpusView bare-string fileid | 934p (base reader) | global fallback (view has no root) | | pl196x | textids index | found by audit | `required_root=self._root` | | mte | MTEFileReader | mvf5 | `required_root` threaded through 8 call sites | | toolbox | StandardFormat.open codecs.open | cr8c | global sandbox (low-level parser) | | named_entity | load_ace_file ann/text | 7qj2 | global sandbox | | nkjp | XML_Tool source file | p4rw class | `required_root=self._root` | `ipipan` already validates via the earlier #3727 fix — unchanged. Fix Each site now calls `nltk.pathsec.validate_path(path, required_root=…)` before opening. Where the reader has a concrete corpus root, the check is scoped with `required_root` (rejects any escape outside that root). `XMLCorpusView` carries no root, so it falls back to the global data-root sandbox via `getattr(self, "_root", None)` — which also avoids an AttributeError on the bare-string path. Reproduced (captured) ``` raw open(symlink) reads: 'TOPSECRET_OUTSIDE_ROOT' BLOCKS the escape validate_path(legit in-root): PASSED <- loads normally ``` Honest residual The global-sandbox fallback (toolbox, named_entity, xmldocs-view) is only as tight as the allowed-roots list, which currently includes the system temp dir. Scoping every reader with `required_root` and removing the temp dir from the allowed roots would harden it further (separate advisory / task). Tests `test_corpus_reader_pathsec.py` — symlink escape rejected, in-root file allowed, XMLCorpusView string-fileid no AttributeError, MTEFileReader out-of-root rejected. 46 existing corpus/toolbox tests pass; all edited modules import (no circular import). pre-commit (black/isort/ruff) clean. --- Scope caveat `validate_path` blocks every symlink escape variant (verified) and equals `pathsec.open()`'s guarantee, but does NOT block hardlinks (no symlink to resolve; tracked separately as GHSA-f794-5jv7-7672) or the validate-then-open TOCTOU race (shared by `pathsec.open`; needs O_NOFOLLOW/openat).