# Spec: PDF heading promotion — script clusters candidates, AI assigns levels, script promotes + verifies

**Status:** Rev 1 — architect Opus 2026-07-29. Implementer = Sonnet. Reviewer ≠ coder.
**Standard:** WCAG 1.3.1 / PDF/UA heading structure. Adds `/H1`–`/H6` tags so a document is navigable.

## Why
veraPDF checks structural *validity*, never whether headings exist. A "digital-tagged" PDF that
passes E3 metadata-patch keeps its original tagging — and real files ship with **everything tagged
`/P`, zero headings** (confirmed: `chapter10-remediated.pdf` = 854 `/P`, 0 `/H`, veraPDF 106/106). The
Ally checker correctly flags "This PDF does not have headings". The E2 re-tag *would* add headings but
mis-detects the callout boxes as 11 tables + 13 figures — worse. So promote headings surgically on the
clean E3 output instead.

## Doctrine — same as figure-alt: script surfaces, AI judges, human approves, script applies+verifies
- **Script** clusters non-body text by font signature and surfaces candidates. It NEVER decides a
  cluster is a heading or its level — font size is evidence, not a verdict (a large word mid-sentence
  isn't a heading; a colored label is).
- **AI** assigns each candidate a heading level (1–6) or "not a heading", from document structure +
  context.
- **Human** approves.
- **Script** promotes the matching `/P` struct elements to `/Hn`, normalizes levels, verifies.

## Deliverables

### 1. `pdf_tree_edit.heading_candidates(pdf, path)` — detector (SCRIPT half)
Scan text spans (via fitz on `path`), compute each span's font signature `(size_rounded, bold,
rgb, font_family)`. The **body** signature = the most common signature among small (<~13pt),
non-bold, dark spans — exclude it. Return candidate occurrences, each:
`{cluster_id, size, bold, rgb, font, text, page, mcids, current_tag, cluster_count}`.
`cluster_id` groups occurrences sharing a signature (so the AI can assign one level to a whole
cluster). `mcids` + page resolve the owning struct element (reuse the figure-alt/`demote` MCID→struct
resolution). Never emit a `level` — that's the AI's.

### 2. `pdf_tree_edit.promote_headings(pdf, assignments)` — promoter (SCRIPT half)
`assignments`: `[{page, mcids, level}]` (or by resolved struct-element). For each, find the `/P` (or
other non-heading) struct element owning those MCIDs and `retag(pdf, elem, "/H{level}")`. Then call
`normalize_heading_levels(pdf)` (clamps level jumps). Skip an assignment whose element can't be
resolved (report it, don't raise). Returns `{promoted: [...], unresolved: [...]}`.

### 3. `scripts/heading_promote.py` — two-phase CLI (mirror `figure_alt.py`)
- `--pdf FILE --build-manifest [--outdir DIR]`: write `heading-manifest.json` =
  candidate occurrences (each with the fields above plus `level: null`). Also emit a compact
  `font_size_rank` summary (distinct signatures sorted largest→smallest) so the AI can see relative
  sizing at a glance. Default `DIR` = `<pdf_stem>.headings/` beside the PDF.
- Agent step (in-session, no script): AI reads the manifest, assigns `level` (1–6) or leaves `null`
  ("not a heading") per occurrence — batching by `cluster_id` is expected.
- `--pdf FILE --apply --manifest FILE`: back up the PDF, `promote_headings` for every occurrence with
  a non-null level, `normalize_heading_levels`, save, re-verify: **veraPDF still compliant AND heading
  count > 0**. Restore backup on any failure. Print result JSON.

## Honesty rules (enforce)
- A candidate left at `level: null` is NOT promoted — never guessed from font size alone.
- Post-apply: veraPDF compliant AND at least one heading present, else restore backup + report.
- Verify text content (`get_text("text")`) byte-identical and a rendered-page pixel spot check
  identical — `retag` changes only `/S` in the struct tree, nothing visual.

## Integration
Document in `skills/file-remediation/SKILL.md`: when a remediated PDF has zero heading tags (or an
Ally "no headings" flag), run this stage — script clusters, AI assigns levels, human approves, apply.
No change to `remediate_pdf.py` routing in this pass (heading promotion is a post-tagging stage on the
tagged output, index-stable, like figure-alt).

## Tests (`tests/test_heading_promote.py` — synthetic PyMuPDF fixtures)
- AC1: `heading_candidates` clusters non-body text, excludes the body signature, emits `level: null`,
  resolves `mcids`.
- AC2: `promote_headings` retags the named `/P`→`/Hn`; `normalize_heading_levels` prevents a level
  jump (H1→H3 becomes H1→H2); veraPDF-representative struct still valid.
- AC3-honesty: null-level candidate not promoted; unresolved mcids reported not raised; apply with 0
  resulting headings (all null) → restore + report, not a false success.
- AC5-fidelity: retag changes only `/S`; `get_text` byte-identical.

`pytest tests/test_heading_promote.py -q` + full `pytest -q` green pre-merge. Reviewer verifies on the
real chapter10 E3 output end-to-end (build → assign H1/H2/H3 → apply → veraPDF compliant + headings
present + text/pixels unchanged).

## Acceptance (real, reviewer verifies; user confirms in Ally)
chapter10 clean E3 output: assign title→H1, "10.x" sections + "In this Chapter"→H2, callout labels
("Explore/Learn/Practice")→H3; after apply, heading tags present (was 0), veraPDF 106/106, reading
order + text + pixels unchanged. Note honestly: final "has headings / 100%" proof is the user
re-uploading to Ally.

## Ponytail floor
Reuse `retag`, `normalize_heading_levels`, the figure-alt MCID→struct resolution, `verapdf_gate`,
backup. New surface = the font-cluster detector + the two-phase CLI + manifest schema. No new
dependency, no ML, no heading heuristic that promotes without the AI's level.
