From 6ed2dfc42fcfd46e287e6fb10f7038e7fb946e5f Mon Sep 17 00:00:00 2001 From: Luis Cosio Date: Sun, 19 Jul 2026 16:38:06 -0700 Subject: [PATCH 1/2] =?UTF-8?q?=E2=AC=86=EF=B8=8F=20chore(deps):=20upgrade?= =?UTF-8?q?=20all=20dependencies=20to=20latest=20versions?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PyMuPDF >=1.23.0 -> >=1.28.0 (resolves to 1.28.0, MuPDF 1.29.0). This is the only runtime dependency; there are no transitive deps. No code changes were required. The extractors use a small, stable slice of the API (fitz.open, page.get_text, page.find_tables, page.number, doc.close, Document indexing/slicing), all unchanged across 1.23 -> 1.28. The `import fitz` alias still works on 1.28. Verified by re-running both PDF extractors against the committed source PDFs and diffing the output: extracted_cnssi_1253.json (1189 controls) and extracted_classified_information.json are byte-identical to the JSON already in the repo. Also gitignores .venv so the local uv environment is never committed. --- .gitignore | 3 ++- requirements.txt | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/.gitignore b/.gitignore index f749f52..bfdf579 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ .DS_Store -__pycache__ \ No newline at end of file +__pycache__ +.venv \ No newline at end of file diff --git a/requirements.txt b/requirements.txt index 0f274e6..fe5fa39 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,3 +1,3 @@ # PDF extraction dependencies -PyMuPDF>=1.23.0 +PyMuPDF>=1.28.0 From 59299208fa84af3cd558e9bbb2981d57e6ba43de Mon Sep 17 00:00:00 2001 From: Luis Cosio Date: Sun, 19 Jul 2026 18:14:29 -0700 Subject: [PATCH 2/2] =?UTF-8?q?=E2=AC=86=EF=B8=8F=20chore:=20standardize?= =?UTF-8?q?=20on=20Python=203.13?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Org-wide standardization on Python 3.13 (3.13 rather than 3.14 because whisperx in a sibling repo caps at <3.14). - Add .python-version pinning 3.13 - README + CLAUDE.md: document 3.13 and switch setup/run commands to uv - Note the extractors' write-to-CWD behavior so verification runs don't clobber the committed JSON No pyproject.toml exists here, so there is no requires-python to change. Verified on 3.13: PyMuPDF 1.28.0 resolves; both extractors regenerate output byte-identical to the committed JSON (diff clean, sha256 match); nist_sorter reports ordering already correct; all 11 tracked .py files compile. No test suite and no CI exist in this repo. --- .python-version | 1 + CLAUDE.md | 22 +++++++++++++++++----- README.md | 19 +++++++++++++------ 3 files changed, 31 insertions(+), 11 deletions(-) create mode 100644 .python-version diff --git a/.python-version b/.python-version new file mode 100644 index 0000000..24ee5b1 --- /dev/null +++ b/.python-version @@ -0,0 +1 @@ +3.13 diff --git a/CLAUDE.md b/CLAUDE.md index d92bde5..a1ceaa2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -15,18 +15,30 @@ This is a web application for viewing and managing security control overlays tha ## Common Development Commands ### Python Data Extraction + +Python 3.13 (pinned in `.python-version`), managed with `uv`. Never use bare `pip` or conda. + ```bash +# One-time environment setup +uv venv --python 3.13 +uv pip install -r requirements.txt + # Extract overlay data from PDFs (requires PyMuPDF/fitz) -python cnssi_1253/extract_cnssi_1253.py cnssi_1253/CNSSI_1253_2022.pdf -python classified_information/extract_classified_information.py classified_information/classified_information_overlay_2022.pdf +uv run python cnssi_1253/extract_cnssi_1253.py cnssi_1253/CNSSI_1253_2022.pdf +uv run python classified_information/extract_classified_information.py classified_information/classified_information_overlay_2022.pdf # Debug specific pages -python cnssi_1253/extract_cnssi_1253.py cnssi_1253/CNSSI_1253_2022.pdf --debug-page 10 +uv run python cnssi_1253/extract_cnssi_1253.py cnssi_1253/CNSSI_1253_2022.pdf --debug-page 10 # Sort NIST controls naturally -python nist_catalog/nist_sorter.py input.json output.json +uv run python nist_catalog/nist_sorter.py input.json output.json ``` +**Extractor output paths:** `extract_*.py` write their JSON to the *current working directory* using hardcoded +filenames, so run them from the repo root to land on the committed files. `nist_sorter.py` overwrites its input +file when no output path is given. When regenerating output just to verify a change, run from a scratch +directory so the committed JSON is never clobbered, then `diff` the result. + ### Development - No build process - edit `index.html` directly - No package manager - pure vanilla JavaScript @@ -93,6 +105,6 @@ No automated tests exist. Manual testing process: ## Dependencies -- **Python**: PyMuPDF (fitz) for PDF extraction +- **Python**: 3.13, PyMuPDF (fitz) for PDF extraction (see `requirements.txt`) - **JavaScript**: None (vanilla JS only) - **Deployment**: GitHub Pages \ No newline at end of file diff --git a/README.md b/README.md index d529d86..082ab14 100644 --- a/README.md +++ b/README.md @@ -108,19 +108,26 @@ The application uses a PDF → JSON → Web pipeline: ### Extracting Data from PDFs +The extractors target **Python 3.13** (pinned in `.python-version`) and depend on PyMuPDF (fitz). Set up the environment with [uv](https://docs.astral.sh/uv/): + ```bash -# Requires PyMuPDF (fitz) -pip install PyMuPDF +# Create the environment and install dependencies +uv venv --python 3.13 +uv pip install -r requirements.txt +``` +The extractor scripts write their JSON output to the **current working directory**, so run them from the repository root: + +```bash # Extract CNSSI 1253 overlay -python cnssi_1253/extract_cnssi_1253.py cnssi_1253/CNSSI_1253_2022.pdf +uv run python cnssi_1253/extract_cnssi_1253.py cnssi_1253/CNSSI_1253_2022.pdf # Extract Classified Information overlay -python classified_information/extract_classified_information.py \ +uv run python classified_information/extract_classified_information.py \ classified_information/classified_information_overlay_2022.pdf # Debug specific pages -python cnssi_1253/extract_cnssi_1253.py cnssi_1253/CNSSI_1253_2022.pdf --debug-page 10 +uv run python cnssi_1253/extract_cnssi_1253.py cnssi_1253/CNSSI_1253_2022.pdf --debug-page 10 ``` --- @@ -192,7 +199,7 @@ The site is deployed at [https://il6.sl5taskforce.org/](https://il6.sl5taskforce | Component | Dependency | |-----------|------------| | **Web App** | None (vanilla JavaScript) | -| **PDF Extraction** | Python 3, PyMuPDF (`pip install PyMuPDF`) | +| **PDF Extraction** | Python 3.13, PyMuPDF (`uv pip install -r requirements.txt`) | | **Deployment** | GitHub Pages | ---