Skip to content

0007. PDF report ingestion: pdfplumber for extraction, a text seam to LLM structuring

Date: 2026-07-13 Status: amended by 0047

Context

M2 ("report understanding") adds FR-01 — ingest a PDF pentest report — which feeds FR-03, the LLM extraction of structured findings. Two things must be fixed before writing code:

  1. Which PDF library turns report bytes into text. Pentest reports are PDFs with headings, per-finding sections, and tables (severity/CWE/endpoint grids are common). FR-01 must tolerate those layouts and reject a malformed/non-report PDF with a clear error, not a crash. Licensing matters: this is a public thesis artifact (Beerware), so a copyleft dependency would contaminate the deliverable.
  2. Where the FR-01 / FR-03 boundary sits. FR-01 is deterministic; FR-03 involves the (non-deterministic, network-bound) LLM. A clean seam keeps FR-01 unit-testable offline and keeps the model's job — and token budget — narrow.

Decision

We will use pdfplumber (MIT) for PDF-to-text extraction, and define the FR-01 output as plain text plus best-effort finding candidates, handing semantic structuring to FR-03's LLM.

  • read_pdf(bytes) -> PdfReport extracts per-page and full-document text with pdfplumber. Text inside table cells is surfaced by extract_text() directly, so table-borne finding data survives without separate table-parsing logic.
  • FR-01 is LLM-free and fail-closed. read_pdf raises a clear PdfError on: a non-PDF (cheap %PDF- magic-header check), a structurally corrupt PDF (pdfplumber/pdfminer parse failure), or a PDF with no extractable text (scanned/image-only — out of scope, no OCR).
  • segment_findings(report) -> tuple[FindingCandidate, ...] splits text on common finding-heading conventions ("Finding N", "F-01", …) into raw candidates; when no heading matches it yields the whole document as one candidate. This is deliberately a best-effort heuristic, not a parser — semantic extraction is the LLM's job (FR-03), not a brittle regex here.
  • The seam: FR-01 emits text/candidates; FR-03 (Pydantic AI, per ADR-0002) turns that into schema-validated Findings. The model never sees raw PDF bytes.

Alternatives considered

  • PyMuPDF / fitz — best-in-class extraction, but AGPL-3.0 (or a paid licence). Copyleft on a public thesis deliverable is a licensing hazard; rejected despite quality.
  • pypdf (BSD) — lightweight, but weaker reading-order and no table awareness. The severity/CWE/endpoint grids common in reports would be scrambled or lost, hurting the FR-03 ≥90% extraction target. Rejected.
  • Explicit table extraction (extract_tables) merged into the text — more code and a real duplication risk (cell text appearing twice). extract_text() already surfaces cell text adequately for LLM consumption; deferred until fidelity proves insufficient.
  • Feed the raw PDF to the LLM / let it segment — couples FR-01 to the model, is un-testable offline, and spends tokens on TOC/boilerplate. Rejected: deterministic extraction first.

Consequences

  • Easier: FR-01 is fully testable with zero network/LLM; malformed input fails closed with clear messages (satisfies FR-01's rejection criterion). pdfplumber's transitive deps (pdfminer.six, Pillow, pypdfium2) are all permissively licensed.
  • Harder / accepted debt: extract_text() flattens complex multi-column tables; exotic layouts may later need explicit table handling or FR-03 prompt robustness. Scanned/image-only PDFs are rejected, not OCR'd — documented as future work.
  • Fixture: a synthetic tests/data/juice_shop_report_synthetic.pdf (generated by scripts/gen_fixture_pdf.py) stands in for now. FR-01's acceptance box is only fully ticked once Álvaro drops in his real Juice Shop report.
  • Status proposed: pending Álvaro's acceptance in async review — the library choice is his call to ratify.