diff --git a/.gitignore b/.gitignore index f749f52..bfdf579 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ .DS_Store -__pycache__ \ No newline at end of file +__pycache__ +.venv \ No newline at end of file diff --git a/.python-version b/.python-version new file mode 100644 index 0000000..24ee5b1 --- /dev/null +++ b/.python-version @@ -0,0 +1 @@ +3.13 diff --git a/CLAUDE.md b/CLAUDE.md index d92bde5..a1ceaa2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -15,18 +15,30 @@ This is a web application for viewing and managing security control overlays tha ## Common Development Commands ### Python Data Extraction + +Python 3.13 (pinned in `.python-version`), managed with `uv`. Never use bare `pip` or conda. + ```bash +# One-time environment setup +uv venv --python 3.13 +uv pip install -r requirements.txt + # Extract overlay data from PDFs (requires PyMuPDF/fitz) -python cnssi_1253/extract_cnssi_1253.py cnssi_1253/CNSSI_1253_2022.pdf -python classified_information/extract_classified_information.py classified_information/classified_information_overlay_2022.pdf +uv run python cnssi_1253/extract_cnssi_1253.py cnssi_1253/CNSSI_1253_2022.pdf +uv run python classified_information/extract_classified_information.py classified_information/classified_information_overlay_2022.pdf # Debug specific pages -python cnssi_1253/extract_cnssi_1253.py cnssi_1253/CNSSI_1253_2022.pdf --debug-page 10 +uv run python cnssi_1253/extract_cnssi_1253.py cnssi_1253/CNSSI_1253_2022.pdf --debug-page 10 # Sort NIST controls naturally -python nist_catalog/nist_sorter.py input.json output.json +uv run python nist_catalog/nist_sorter.py input.json output.json ``` +**Extractor output paths:** `extract_*.py` write their JSON to the *current working directory* using hardcoded +filenames, so run them from the repo root to land on the committed files. `nist_sorter.py` overwrites its input +file when no output path is given. When regenerating output just to verify a change, run from a scratch +directory so the committed JSON is never clobbered, then `diff` the result. + ### Development - No build process - edit `index.html` directly - No package manager - pure vanilla JavaScript @@ -93,6 +105,6 @@ No automated tests exist. Manual testing process: ## Dependencies -- **Python**: PyMuPDF (fitz) for PDF extraction +- **Python**: 3.13, PyMuPDF (fitz) for PDF extraction (see `requirements.txt`) - **JavaScript**: None (vanilla JS only) - **Deployment**: GitHub Pages \ No newline at end of file diff --git a/README.md b/README.md index d529d86..082ab14 100644 --- a/README.md +++ b/README.md @@ -108,19 +108,26 @@ The application uses a PDF → JSON → Web pipeline: ### Extracting Data from PDFs +The extractors target **Python 3.13** (pinned in `.python-version`) and depend on PyMuPDF (fitz). Set up the environment with [uv](https://docs.astral.sh/uv/): + ```bash -# Requires PyMuPDF (fitz) -pip install PyMuPDF +# Create the environment and install dependencies +uv venv --python 3.13 +uv pip install -r requirements.txt +``` +The extractor scripts write their JSON output to the **current working directory**, so run them from the repository root: + +```bash # Extract CNSSI 1253 overlay -python cnssi_1253/extract_cnssi_1253.py cnssi_1253/CNSSI_1253_2022.pdf +uv run python cnssi_1253/extract_cnssi_1253.py cnssi_1253/CNSSI_1253_2022.pdf # Extract Classified Information overlay -python classified_information/extract_classified_information.py \ +uv run python classified_information/extract_classified_information.py \ classified_information/classified_information_overlay_2022.pdf # Debug specific pages -python cnssi_1253/extract_cnssi_1253.py cnssi_1253/CNSSI_1253_2022.pdf --debug-page 10 +uv run python cnssi_1253/extract_cnssi_1253.py cnssi_1253/CNSSI_1253_2022.pdf --debug-page 10 ``` --- @@ -192,7 +199,7 @@ The site is deployed at [https://il6.sl5taskforce.org/](https://il6.sl5taskforce | Component | Dependency | |-----------|------------| | **Web App** | None (vanilla JavaScript) | -| **PDF Extraction** | Python 3, PyMuPDF (`pip install PyMuPDF`) | +| **PDF Extraction** | Python 3.13, PyMuPDF (`uv pip install -r requirements.txt`) | | **Deployment** | GitHub Pages | --- diff --git a/requirements.txt b/requirements.txt index 0f274e6..fe5fa39 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,3 +1,3 @@ # PDF extraction dependencies -PyMuPDF>=1.23.0 +PyMuPDF>=1.28.0