Skip to content

0037. Enrich ingested findings with CVSS and MITRE ATT&CK, inferring when absent

Date: 2026-07-20

Status: accepted

Context

The state-of-the-art review (thesis §2.1.3) argues that a revalidation system "must map free-text findings onto" the standard reference frameworks — CVSS for severity and MITRE ATT&CK for the attacker behaviour a finding enables — "to reason about what a finding is and how it should be re-verified." Until now the ingestion pipeline (FR-03, extract.py) captured a finding's prose fields (description, impact, attack vector, endpoints, reproduction steps) but not its CVSS code or ATT&CK mapping, so that claim was unrealised in the artefact.

Real pentest reports state these frameworks inconsistently: some findings carry a CVSS vector and score, many do not; ATT&CK technique IDs are rarely written out at all. The lab reports used for evaluation (e.g. the OWASP Juice Shop report) mostly omit them. Extracting "only what the report states" (the pipeline's standing rule) would therefore leave the new fields empty on most findings — no better than not having them.

Decision

Add a CVSS code and a MITRE ATT&CK technique mapping to every ingested finding, and derive them from the finding when the report is silent.

  • New frozen domain models CvssCode (vector, base_score, inferred) and MitreMapping (techniques, inferred) in domain.py; Finding gains cvss and mitre, both defaulting to empty.
  • ExtractedFinding gains the same two fields (defaulted), and the extraction instructions tell the model to copy a stated CVSS/ATT&CK verbatim with inferred=false, or derive a best-estimate CVSS v3.1 base vector/score and the most applicable ATT&CK technique IDs with inferred=true when the report states none.
  • Persistence: FindingVersionRecord gains cvss and mitre JSON columns, with from_domain/to_domain round-tripping them, so the mapping survives as first-class data (not only inside the raw audit blob).

Provenance is explicit. The inferred flag records whether each code was read from the report or generated by the model, so a downstream reader (and the operator) can always tell a stated value from an estimate.

This is a deliberate, bounded exception to the "never guess" extraction rule (FR-03's schema-validation gate, ADR-0009). It is justified because CVSS and ATT&CK are classificatory metadata — severity and technique labels — not verdict evidence: an inferred CVSS score never feeds the still-open/fixed/inconclusive determination, which continues to rest only on re-executed reproduction steps and observed output. Inferring a label therefore cannot make a verdict less conservative, and the label is marked as inferred either way.

Alternatives considered

  • Extract only when the report states them. Rejected: most lab and real reports omit CVSS vectors and almost never write ATT&CK IDs, so the fields would be empty on the majority of findings and the §2.1.3 mapping would stay aspirational. The whole value here is a complete mapping the operator can lean on and correct.
  • Map to CWE instead of / in addition to ATT&CK. Deferred. ATT&CK was chosen first because the thesis frames it as the finding→attacker-behaviour link, and because "MITRE" was the framework named in the request. CWE (the weakness class) is a natural, cheap future addition alongside cvss/mitre if wanted.
  • Compute CVSS from a metrics vector locally rather than let the model score it. Rejected for now: it adds a scoring dependency for marginal gain over a model that already knows the CVSS formula, and the score is advisory (marked inferred), not a gate.

Consequences

  • The §2.1.3 taxonomy claim is now realised in the artefact: every finding carries a severity code and a technique mapping, extracted or estimated.
  • Inferred codes are model estimates. They are labelled inferred=true and are the operator's to accept or correct (they ride the FR-16 edit lifecycle like any other finding field). They must not be read as authoritative severities.
  • Verdict conservatism is untouched. The taxonomy fields are metadata; the retest verdict still depends only on re-executed evidence.
  • No migration (ADR-0008): the columns are created by create_all on a fresh database; a pre-existing local revalid.db is recreated (rm) rather than altered.
  • Surfacing is staged. This slice covers ingestion + persistence + tests; the API Finding output and the SPA finding view still need to expose the two fields (follow-up), as does tagging the evaluation ground truth with them.

Update (2026-07-25, issue #226) — the operator sets the taxonomy by hand

The consequence above said inferred codes "ride the FR-16 edit lifecycle like any other finding field". They did not. FindingEditIn.to_finding() never carried cvss/mitre across, so both fell back to their empty defaults and every operator edit silently destroyed the taxonomy — the exact opposite of the "operator's to accept or correct" claim.

Worse, correcting was impossible in the first place: only the extraction path ever populates these fields. A finding that arrived through the FR-02 structured importer or through manual entry (ADR-0020) had none, and nothing backfilled it. For a manually-entered corpus — which is how the FR-15 evaluation was ingested — the taxonomy was simply unreachable.

Decision. The finding editor owns the taxonomy. FindingEditIn gains optional cvss and mitre objects: omitted leaves the current values untouched, supplied sets them. This is the one place a CVSS code or technique mapping can be entered by hand, which makes it the answer for every non-extraction ingestion path.

Provenance stays the server's to decide, and the client cannot assert it. The request carries no inferred flag. The server compares the submitted value against the current version:

  • omitted → keep the current value, provenance included;
  • supplied and identical → likewise. This matters: the editor always posts the form's contents, so a round-trip of an untouched inferred value must not launder a model guess into an author-stated fact;
  • supplied and different → the operator authored it, so inferred becomes false.

This preserves the property the original decision rests on — that a reader can always tell an estimate from a stated value — while making the correction path real. The rest of ADR-0037 is unchanged: the taxonomy remains classificatory metadata that never feeds a verdict, which is what justified inferring it at all.

Update — 2026-07-25: enrichment reaches the LLM-free doors, opt-in (issue #233)

The update above made the taxonomy correctable everywhere but left it derivable only on the PDF door, because enrichment is part of the extraction call and the FR-02 / manual doors make no call. Álvaro asked for derivation on those doors too. The tension is that both are deliberately LLM-free — that property is why manual entry is the recommended seeding path for demos and tests (deterministic, instant, free), and ADR-0020 exists precisely because a model could not be relied on there.

Decision: an opt-in flag, default off. POST /api/findings/import?enrich=true and {"enrich": true} on POST /api/reports/manual run a taxonomy pass — one build_taxonomy_agent call per finding, filling only what the source left empty. Omitted or false, no agent is invoked at all, so the LLM-free guarantee holds as an absolute rather than as a promise about speed. The SPA surfaces it as a checkbox stating the cost ("one model call per finding"), unticked by default, and the toggle overrides any enrich pasted into the JSON editor so what the operator can see is what runs.

Three properties carry over from the decision above rather than being re-argued:

  • Copying is not inferring. A stated cvssv3/cvssv3_score in a DefectDojo export is mapped across on every import, flag or no flag, with inferred=false. It needs no model, so it is not gated behind one.
  • A stated CWE is not an ATT&CK technique. DefectDojo carries cwe; mapping a weakness id onto a technique id would fabricate a claim, so it is left empty for the enrichment pass (or the operator) to fill honestly.
  • Provenance stays server-stamped. FindingTaxonomy — the enrichment model's output schema — carries no inferred field at all, so the model has no way to express "the source stated this"; everything it produces is recorded inferred=true, and a value already stated is never overwritten.

Consequence. A failed enrichment call costs that finding its taxonomy, never the import: the finding persists unchanged and the response reports enrichment_failed, so a partially-enriched import cannot be mistaken for a complete one. The FR-15 corpus remains as it was ingested — no taxonomy — because re-running it with the flag would change the evaluated artefact, which is the author's call, not a side effect of this change.